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Abstract 

In  the  past  decades  an  increasing  number  of  problems  in  continuum  theory  have  been 
treated  using  stochastic  dynamical  theories.  This  is  because  dynamical  systems  governing 
real  processes  always  contain  some  elements  characterized  by  uncertainty  or  stochasticity. 
Uncertainties  may  arise  in  the  system  parameters,  the  boundary  and  initial  conditions,  and 
also  in  the  external  forcing  processes.  Also,  many  problems  are  treated  through  the  stochas¬ 
tic  framework  due  to  the  incomplete  or  partial  understanding  of  the  governing  physical  laws. 
In  all  of  the  above  cases  the  existence  of  random  perturbations,  combined  with  the  com¬ 
plex  dynamical  mechanisms  of  the  system  often  leads  to  their  rapid  growth  which  causes 
distribution  of  energy  to  a  broadband  spectrum  of  scales  both  in  space  and  time,  making 
the  system  state  particularly  complex.  Such  problems  are  mainly  described  by  Stochastic 
Partial  Differential  Equations  and  they  arise  in  a  number  of  areas  including  fluid  mechanics, 
elasticity,  and  wave  theory,  describing  phenomena  such  as  turbulence,  random  vibrations, 
flow  through  porous  media,  and  wave  propagation  through  random  media.  This  is  but  a 
partial  listing  of  applications  and  it  is  clear  that  almost  any  phenomenon  described  by  a 
field  equation  has  an  important  subclass  of  problems  that  may  profitably  be  treated  from 
a  stochastic  point  of  view. 

In  this  work,  we  develop  a  new  methodology  for  the  representation  and  evolution  of 
the  complete  probabilistic  response  of  infinite-dimensional,  random,  dynamical  systems. 
More  specifically,  we  derive  an  exact,  closed  set  of  evolution  equations  for  general  nonlinear 
continuous  stochastic  fields  described  by  a  Stochastic  Partial  Differential  Equation.  The 
derivation  is  based  on  a  novel  condition,  the  Dynamical  Orthogonality  (DO),  on  the  repre¬ 
sentation  of  the  solution.  This  condition  is  the  ‘key’  to  overcome  the  redundancy  issues  of 
the  full  representation  used  while  it  does  not  restrict  its  generic  features.  Based  on  the  DO 
condition  we  derive  a  system  of  field  equations  consisting  of  a  Partial  Differential  Equation 
(PDE)  for  the  mean  field,  a  family  of  PDEs  for  the  orthonormal  basis  that  describe  the 
stochastic  subspace  where  uncertainty  ‘lives’  as  well  as  a  system  of  Stochastic  Differential 
Equations  that  defines  how  the  uncertainty  evolves  in  the  time  varying  stochastic  subspace. 
If  additional  restrictions  are  assumed  on  the  form  of  the  representation,  we  recover  both 
the  Proper-Orthogonal-Decomposition  (POD)  equations  and  the  generalized  Polynomial- 
Chaos  (PC)  equations;  thus  the  new  methodology  generalizes  these  two  approaches.  For 
the  efficient  treatment  of  the  strongly  transient  character  on  the  systems  described  above  we 
derive  adaptive  criteria  for  the  variation  of  the  stochastic  dimensionality  that  characterizes 
the  system  response.  Those  criteria  follow  directly  from  the  dynamical  equations  describing 
the  system. 

We  illustrate  and  validate  this  novel  technique  by  solving  the  2D  stochastic  Navier-Stokes 
equations  in  various  geometries  and  compare  with  direct  Monte  Carlo  simulations.  We  also 
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apply  the  derived  framework  for  the  study  of  the  statistical  responses  of  an  idealized  ‘double 
gyre’  model,  which  has  elements  of  ocean,  atmospheric  and  climate  instability  behaviors. 

Finally,  we  use  our  new  stochastic  description  for  flow  fields  to  study  the  motion  of 
inertial  particles  in  flows  with  uncertainties.  Inertial  or  finite-size  particles  in  fluid  flows  are 
commonly  encountered  in  nature  (e.g.,  contaminant  dispersion  in  the  ocean  and  atmosphere) 
as  well  as  in  technological  applications  (e.g.,  chemical  systems  involving  particulate  reactant 
mixing).  As  it  has  been  observed  both  numerically  and  experimentally,  their  dynamics 
can  differ  markedly  from  infinitesimal  particle  dynamics.  Here  we  use  recent  results  from 
stochastic  singular  perturbation  theory  in  combination  with  the  DO  representation  of  the 
random  flow,  in  order  to  derive  a  reduced  order  inertial  equation  that  will  describe  efficiently 
the  stochastic  dynamics  of  inertial  particles  in  arbitrary  random  flows. 

Thesis  Supervisor:  Pierre  F.J.  Lermusiaux 
Title:  Associate  Professor 


4 


Acknowledgments 


I  wish  to  extend  warm  thanks  to  my  advisors  Pierre  Lermusiaux  and  George  Haller  for 
this  wonderful  journey.  It  all  started  four  years  ago  when  I  began  to  work  with  George 
on  the  nonlinear  dynamics  of  small  particles.  A  year  after  I  joined  MSEAS  group  and  I 
start  my  research  on  stochastic  flows  with  Pierre.  I  am  indebted  to  both  of  them  for  giving 
me  a  well  balanced  mix  of  freedom  and  guidance  on  my  research,  allowing  me  to  put  time 
on  new  ideas  while  maintaining  the  scientific  ‘entropy’  on  reasonable  and  efficient  levels. 
Special  thanks  to  George  for  doing  that  remotely  with  such  an  enthusiasm  but  also  for 
teaching  me  to  present  results  in  a  rigorous  but  nevertheless  attractive  way.  Special  thanks 
to  Pierre  for  teaching  me  that  mathematical  modeling  can  and  should  always  be  performed 
in  connection  with  physical  intuition  and  understanding  of  the  considered  phenomenon.  I 
am  also  grateful  to  the  members  of  my  thesis  committee,  Dennis  McLaughlin,  Nicholas 
Patrikalakis,  and  Carl  Wunsch  for  their  constructive  comments  and  advise. 

I  owe  a  huge  debt  of  gratitude  to  Alex  Vakakis  for  his  advise  and  support  especially 
during  some  difficult  times  of  my  graduate  life.  I  met  him  many  years  ago  when  I  took  his 
Nonlinear  Dynamics  classes  as  undergrad  in  Greece  and  since  then  he  has  been  a  mentor, 
friend,  and  collaborator.  I  am  grateful  to  my  diploma  thesis  advisor  Makis  Athanassoulis 
whose  classes,  seminars,  and  endless  hours  put  on  joint  research  on  various  applied  math 
topics  and  especially  on  stochastic  systems  were  important  background  for  the  work  done 
in  this  thesis. 

I  wish  to  extend  warm  thanks  to  my  officenrate  and  friend,  Matt  Ueckermann,  for  his  help 
and  support  on  the  numerical  part  of  the  thesis.  Matt’s  enthusiasm  on  scientific  computing 
led  to  a  two-three  orders  of  magnitude  speed  increase  of  the  original  codes.  This  was  not 
enough  for  him  though  so  he  implemented  the  developed  algorithms  using  his  finite-volume 
framework,  ending  up  with  an  even  faster  and  more  accurate  code  which  was  used  for  some 
of  the  results  included  in  this  work. 

I  feel  indebted  to  all  my  collaborators:  Larry  Bergman  for  collaboration  on  random 
vibrations  and  also  his  advise  and  support,  Jerry  Gollub  and  Nick  Ouellette  for  work¬ 
ing  on  experimental  data  analysis  for  particles,  Jeff  Peng  and  John  Dabiri  for  providing 
data  and  collaborating  on  the  predator-prey  interactions  problem  around  a  jellyfish,  Sotiria 
Koloutsou- Vakakis  for  collaboration  on  transportation  of  pollutants,  and  Raf  Ferrari  for 
discussions  and  collaboration  on  ocean  mixing  and  Lagrangian  Coherent  Structures. 

Thanks  also  to  all  the  members  in  the  Nonlinear  Dynamics  Lab  and  the  MSEAS  group 
for  four  wonderful  years.  Special  thanks  to  my  officemates  who  helped  create  a  great  work¬ 
ing  environment:  Mani  Mathur,  Wenbo  Tang,  Lisa  Burton,  Eric  Heubel,  Arpit  Agarwal, 
Thomas  Sondergaard,  Konur  Yigit,  Tapovan  Lolla,  Latifah  Harnzah,  and  Melissa  Kaufman. 
I  also  want  to  thank  Leslie  Regan  and  Marcia  Munger  for  solving  every  administrative 
problem. 

I  would  like  to  acknowledge  ONR  (N00014-07-1-1061,  N00014-08-1-1097,  N00014-07-1- 
0241  (QPE)),  AFOSR  (FA  9550-06-0092),  and  NSF  (DMS-04-  04845)  for  supporting  my 
research.  I  have  very  much  appreciated  the  opportunity  to  attend  many  conferences  and 
meetings;  discussing  my  work  with  other  scientists  has  been  an  important  part  of  my  grad¬ 
uate  career.  I  especially  thanks  MIT  for  awarding  me  the  Presidential  Fellowship  ‘George 
and  Marie  Vergottis’  for  the  first  two  years. 

I  also  want  to  show  my  gratitude  to  Kirnon  Bekelis  and  George  Gikas  for  endless  hours 
of  discussions  on  academic  and  non- academic  issues,  and  for  always  being  there  as  friends 
whenever  I  need  them. 


5 


This  list  would  not  be  complete  without  a  special  thanks  to  my  family  here  at  US,  uncle 
Foti,  aunt  Laura,  Sonia,  Rob  and  Alex  for  their  unconditional  love  and  support  in  many 
situations.  Last  but  not  least,  I  am  grateful  to  my  parents  and  my  brother  Panteli.  I  have 
no  way  to  thank  them  for  their  invaluable  role  in  my  life  but  to  dedicate  this  dissertation 
to  them. 


6 


Contents 


1  Introduction  17 

1.1  Preview  of  chapters .  18 

2  Representations  of  stochastic  fields  21 

2.1  Introduction .  21 

2.2  The  family  of  probability  density  functions  .  23 

2.3  The  moment  system .  25 

2.3.1  Connection  with  the  family  of  characteristic  functions .  26 

2.4  The  characteristic  functional  .  27 

2.4.1  Determination  of  the  moments  system .  28 

2.4.2  Determination  of  the  family  of  characteristic  functions .  29 

2.4.3  Characteristic  functionals  with  explicit  description .  29 

2.5  The  Karhunen  Loeve  expansion .  30 

2.5.1  A  geometrical  interpretation  .  31 

2.5.2  Application  to  fluid  flows .  33 

3  Evolution  of  stochastic  fields  37 

3.1  Introduction .  37 

3.2  Definitions  and  problem  statement .  40 

3.3  The  stochastic  subspace  and  the  dynamical  orthogonality  condition .  41 

3.4  Dynamically  orthogonal  field  equations .  43 

3.4.1  The  case  of  independent  increment  excitation  (white  noise) .  46 

3.5  Consistency  with  existing  methodologies .  49 

3.5.1  Generalized  polynomial  chaos  expansion .  49 

3.5.2  Proper  orthogonal  decomposition .  50 

4  Evolution  of  the  stochastic  dimensionality  51 

4.1  Introduction .  51 

4.2  Cost  scaling  with  the  stochastic  dimensionality .  52 

4.3  Update  of  the  stochastic  subspace  using  stability  properties  of  the  SPDE  .  .  53 

4.3.1  Conditions  for  the  evolution  of  the  stochastic  dimensionality  ....  54 

4.3.2  Selection  of  new  stochastic  dimensions .  55 

4.4  Update  of  the  stochastic  subspace  using  data  and  measurements .  58 

4.4.1  Data  and  measurements  formulation .  58 

4.4.2  Update  of  the  stochastic  information  inside  V5 .  59 

4.4.3  Expansion  of  the  stochastic  subspace  V5  .  61 


7 


5  Application  of  dynamically  orthogonal  field  equations  to  random  fluid 

flows  63 

5.1  Formulation .  63 

5.2  Derivation  of  DO  Navier-Stokes  equations .  65 

5.2.1  Stochastic  pressure  field .  65 

5.2.2  Evolution  of  the  mean  field  u  (x,  t;  to)  .  67 

5.2.3  Evolution  of  the  stochastic  subspace  basis  Uj  (x,  t;  u) .  67 

5.2.4  Evolution  of  the  stochastic  coefficients  Y)  (x,  t;  cu) .  68 

5.3  The  case  of  stochastic  boundary  conditions  .  69 

5.4  Unstable  perturbations  for  Navier-Stokes  equations .  70 

5.5  Transfer  of  energy  in  Navier-Stokes .  72 

5.5.1  Energy  exchanges  between  the  DO  modes  and  the  mean  flow  ....  73 

5.5.2  Energy  exchanges  between  the  DO  modes .  74 

5.5.3  Stochastic  energy  in  Navier-Stokes .  75 

5.6  Application  I:  Lid  driven  cavity  flow  with  stochastic  initial  conditions  ...  76 

5.6.1  Evolution  of  a  small  stochastic  perturbation .  79 

5.6.2  Convergence  with  respect  to  the  stochastic  dimensionality .  83 

5.7  Application  II:  Flow  past  a  circular  disk  with  stochastic  initial  conditions  .  86 

5.8  Application  III:  Instabilities  in  the  forced  double  gyre  flow  in  a  basin  ....  94 

5.8.1  Model .  97 

5.8.2  An  overview  of  deterministic  analysis .  98 

5.8.3  Bifurcation  analysis  of  the  stochastic  response .  101 

5.8.4  Stochastic  response  for  larger  Reynolds  number .  Ill 

6  Finite-size  particles  in  stochastic  flows  119 

6.1  Introduction .  119 

6.2  Summary  of  results  for  finite-size  particles  in  deterministic  flows  .  120 

6.2.1  Reduced  order  dynamics .  121 

6.2.2  Instabilities  on  the  dynamics  of  finite-size  particles .  127 

6.2.3  Clustering  of  finite-size  particles  .  129 

6.3  Dynamics  of  finite-size  particles  in  stochastic  flows .  131 

6.3.1  Stochastic  velocity  field .  131 

6.3.2  Markov  (diffusion)  approximation  .  132 

6.3.3  Stochastic  slow  manifold .  135 

6.3.4  Stochastic  inertial  equation .  137 

6.3.5  Stochastic  source  inversion  .  139 

6.4  Higher  order  Lagrangian  stochastic  models  .  139 

6.5  Clustering  due  to  stochasticity  of  the  flow .  140 

6.6  Application:  Particles  in  the  double  gyre  flow .  142 

6.6.1  Stochastic  slow  manifold  in  the  flow .  142 

6.6.2  Convergence  to  the  stochastic  slow  manifold .  147 

6.6.3  Validation  of  stochastic  inertial  equation .  147 

6.6.4  Clustering  due  to  the  combined  effect  of  inertial  and  flow  stochasticity  150 

7  Conclusions  155 

7.1  Future  directions .  156 


A  DO  equations  for  3D  Navier-Stokes  in  component  wise  form  159 

A.l  Stochastic  pressure  field .  161 

A. 2  Evolution  of  the  mean  field  u  (x,  t;ui) .  163 

A. 3  Evolution  of  the  stochastic  subspace  basis  Xi  (x,  t',  uj) .  163 

A.  4  Evolution  of  the  stochastic  coefficients  Yt  (x,  t\u) .  165 

B  Numerical  implementation  of  2D  DO  for  Navier-Stokes  167 

B. l  Initial  conditions  formulation .  167 

B.1.1  Storage  of  orthonormal  basis .  168 

B.2  Evolution  of  stochastic  state .  168 

B.3  Diagonalization  of  covariance  matrix  -  Adaptive  criteria .  168 

B. 4  Storage  -  Plotting .  170 

C  Normal  local  stability  of  general  invariant  manifolds  171 

C. l  Set-up  and  definitions .  171 

C.2  Normally  stable  and  normally  unstable  subsets .  172 

C.3  Computing  the  NILE .  175 

C.4  Locating  stable  and  unstable  neighborhoods  of  M(t) .  176 

C.5  Stability  properties  of  non- neutrally  buoyant  particles .  177 


9 


10 


List  of  Figures 


2- 1  Ellipsoid  that  contains  the  main  spread  (‘mass’)  of  the  probability  measure; 

defined  by  the  principal  directions  and  the  associated  eigenvalues  of  the  cor¬ 
relation  operator .  32 

3- 1  a)  The  stochastic  subspace  Vs  spanned  by  fields  u,  (x,  t)  that  corresponds 

to  important  variance,  b)  The  variation  of  the  u;  (x,  t )  inside  Vs  can  always 
be  covered  by  a  rotation  of  the  stochastic  coefficients  Yi(t;u) .  44 

4- 1  Computational  time  (sec)  for  the  lid-driven  cavity  flow  described  in  Chapter 

5,  using  different  number  of  modes  (red  curve).  The  blue  line  indicates  the 
linear  ’best  fit’  in  the  log- log  plot  and  it  has  an  inclination  equal  to  1.986  (2 

is  the  theoretical  prediction) .  54 

4-2  Decomposition  of  the  stochastic  subspace  Vs  based  on  the  data  subspace  Vo-  59 

4- 3  Update  of  the  probability  density  function  /y  (y  ,  i)  using  data  or  measure¬ 

ments .  62 

5- 1  Energy  exchanges  between  DO  modes  and  the  mean  flow  in  stochastic,  ho¬ 

mogeneous,  Navier-Stokes  equations.  The  energy  flow  from  the  mean  to  the 
modes  is  characterized  by  the  second  order  statistics  while  variance  exchange 

among  the  modes  is  characterized  by  the  third  order  statistics .  77 

5-2  Driven  cavity  flow,  problem  configuration .  77 

5-3  Initial  conditions  for  the  mean  and  the  basis  of  the  stochastic  subspace  Vs 

in  terms  of  the  field  streamfunction .  78 

5-4  Evolution  of  principal  variances  crj(t),  i  =  1, ...,  5  (blue  curves)  and  mean 

field  energy  (red  curve)  for  the  flow  in  a  cavity .  79 

5-5  Mean  field  and  basis  of  the  stochastic  subspace  Vs  in  terms  of  the  stream- 
function  and  vorticity  field;  two-dimensional  marginals  of  the  five  dimensional 

joint  pdf  /  (y,  t)  at  time  t  =  2 .  80 

5-6  Mean  field  and  basis  of  the  stochastic  subspace  Vs  in  terms  of  the  stream- 
function  and  vorticity  field;  two-dimensional  marginals  of  the  five  dimensional 

joint  pdf  /  (y ,  t)  at  time  t  =  8 .  81 

5-7  Mean  velocity  field  (streamfunction)  computed  using  Monte-Carlo  method 

(250  and  500  samples)  and  the  DO  field  equations  (s  =  5  modes)  at  t  =  1.  .  82 

5-8  Upper  plots:  mean  flow  for  t  =  0.7  in  terms  of  the  streamfunction  for  i)  the 
four  modes  solution  and  ii)  the  five  modes  solution.  Lower  plot:  the  red 
curves  represent  the  time  series  for  the  variances  af  (t) ,  i  =  1,  ..,5  of  the 
five-modes  solution  v  (x,  t;  lu)  while  the  blue  curves  represent  the  projection 
of  the  four  modes  solution  u  (x,  f;  u>)  to  the  modes  Vj  (x,  t) .  84 


11 


5-9  Upper  plots:  mean  flow  for  t  =  5.98  in  terms  of  the  streamfunction  for  i) 
the  four  modes  solution  and  ii)  the  five  modes  solution.  Lower  plot:  the 
red  curves  represent  the  time  series  for  the  variances  erf  (t) ,  i  =  1, ..,  5  of  the 
five-modes  solution  v  (x,  t;  uj)  while  the  blue  curves  represent  the  projection 

of  the  four  modes  solution  u  (x,  f;  w)  to  the  modes  Vj  (x,  t) .  85 

5-10  Mean  flow  for  various  stochastic  dimensionalities  and  for  t  =  1 .  86 

5-11  Mean  flow  for  various  stochastic  dimensionalities  and  for  t  =  2 .  87 

5-12  Mean  flow  for  various  stochastic  dimensionalities  and  for  t  =  6 .  88 

5-13  Left  column:  mean  flow  for  various  stochastic  dimensionalities  dim  (V,§)  at 
t  =  2.  The  four  right  columns  on  the  right  show  the  four  most  energetic 
modes  Uj  (x,  t)  in  terms  of  their  streamfunction  as  well  as  their  associated 

variance  Ew  [bj2  (f;  tu)]  as  a  function  of  time .  89 

5-14  Left  column:  mean  flow  for  various  stochastic  dimensionalities  dim  (V,§)  at 
t  =  6.  The  four  right  columns  on  the  right  show  the  four  most  energetic 
modes  Uj  (x,  t)  in  terms  of  their  streamfunction  as  well  as  their  associated 

variance  Eu  [Y2  (f;  cu)]  as  a  function  of  time .  90 

5-15  Left  column:  mean  flow  for  various  stochastic  dimensionalities  dim  (V,§)  at 
t  =  8.  The  four  right  columns  on  the  right  show  the  four  most  energetic 
modes  Uj  (x,  t)  in  terms  of  their  streamfunction  as  well  as  their  associated 

variance  Eu  [Y2  (t;  cu)]  as  a  function  of  time .  91 

5-16  Upper  plot:  instantaneous  error  of  the  solution  with  respect  to  the  number 
of  modes  used  and  time.  Lower  plot:  time  averaged  error  with  respect  to  the 

number  of  DO  modes .  92 

5-17  A  typical  realization  of  the  flow  past  a  circular  disk .  92 

5-18  Initial  conditions  for  the  mean  and  the  basis  of  the  stochastic  subspace  Vs 

in  terms  of  the  field  streamfunction .  93 

5-19  Evolution  of  principal  variances  a2  (t) ,  i  =  1, ...,  5  (blue  curves)  and  mean 

field  energy  (red  curve)  for  the  flow  behind  a  disk .  94 

5-20  Mean  field  and  basis  of  the  stochastic  subspace  Vs  in  terms  of  the  stream- 
function;  two-dimensional  marginals  of  the  five  dimensional  joint  pdf  /  (y,  t) 

at  time  t  =  2 .  95 

5-21  Mean  field  and  basis  of  the  stochastic  subspace  Vs  in  terms  of  the  stream- 
function;  two-dimensional  marginals  of  the  five  dimensional  joint  pdf  /  (y,  t) 

at  time  t  =  4 .  96 

5-22  Mean  velocity  field  (streamfunction)  computed  using  the  DO  field  equations 

(s  =  5)  and  Monte-Carlo  method  (500  samples)  at  t  =  1 .  96 

5-23  Bifurcation  diagram  for  the  QG  model  with  values  of  the  parameters  as  in 

Tables  5.1  and  5.2  (from  Simonnet  and  Dijkstra  [133]) .  99 

5-24  Spectral  behavior  of  the  eigenmodes  involved  into  the  various  bifurcations  of 
the  anitsymmetric  branch  for  the  barotropic  QG  model  (from  Simonnet  and 

Dijkstra  [133]) .  100 

5-25  Stochastic  responce  of  the  QG  model  for  Re  =  25.  The  left  top  plot  shows 
the  mean  flow  in  terms  of  the  vorticity  (colormap)  and  the  streamlines  (black 
solid  curves).  The  right-top  plot  indicates  the  energy  of  the  mean  flow  (red 
curve)  and  the  variance  of  the  stochastic  mode.  The  bottom  plot  indicates 
the  first  DO  mode  together  with  the  pdf  for  the  stochastic  coefficient.  .  .  .  101 

5-26  Stochastic  responce  of  the  QG  system  for  Re  =  35  after  the  system  has 

reached  steady  state  statistics .  102 


12 


5-27  Oscillation  of  the  DO  mode  during  convergence  to  the  steady  state  attractor 

for  Re  =  35 .  103 

5-28  Addition  of  an  extra  mode  at  the  time  instant  where  the  existing  mode 

exceeds  the  predefined  variance  (Flow  Re  =  38) .  104 

5-29  Mode  removal  due  to  very  low  variance  (Flow  Re  =  38) .  105 

5-30  The  variance  of  the  stochastic  perturbation  becomes  comparable  with  the 
energy  of  the  mean  flow.  At  this  point  mutual  interactions  between  the 

mean  flow  and  the  perturbation  begin  to  occur .  106 

5-31  Steady  state  regime  for  Re  =  38.  Note  that  the  perturbation  retains  its 
symmetric  character  even  though  the  variance  of  the  mode  is  comparable 

with  energy  of  the  mean  flow .  107 

5-32  Stochastic  response  for  Re  =  40.  Note  that  the  first  mode  has  symmetric 
spatial  properties  which  are  accompanied  by  symmetric  pdf  while  the  second 

mode  has  antisymmetric  spatial  properties  with  non-symmetric  pdf .  108 

5-33  Initial  stage  of  motion  for  Re  =  55 .  109 

5-34  As  time  evolves  more  modes  have  to  be  added  in  order  to  achieve  the  given 

tolerance  (in  terms  of  variance) .  110 

5-35  Convergence  to  a  deterministic  attractor  after  a  transient  stochastic  regime 

for  Re  =  65 .  Ill 

5-36  Response  for  Re  =  85.  In  this  case  the  convergence  to  the  deterministic 
attractor  occurs  earlier  with  the  initial  stochastic  regime  being  much  sorter 

in  time .  112 

5-37  Convergence  to  a  deterministic  attractor  for  Re  =100 .  113 

5-38  Stochastic  attractor  after  a  temporal  convergence  to  deterministic  dynamics 

(Re  =  200) .  114 

5-39  Summary  of  the  stochastic  analysis  for  the  double  gyre  flow  over  various 

Reynolds  numbers .  114 

5-40  Initial  regime  of  the  stochastic  response  for  Re  =  104 .  115 

5-41  The  modes  added  are  localized  around  the  boundary  of  the  formed  gyres  of 

the  mean  flow .  115 

5-42  After  the  mean  flow  energy  exceeds  a  certain  limit  an  instability  breaks  the 

symmetry  of  the  third  mode  as  it  is  shown  in  the  corresponding  pdf  plot.  .  116 

5- 43  The  instabilitiy  shown  in  the  previous  figure  is  the  starting  point  for  the 

non-Gaussian  statistics  shown  here .  117 

6- 1  (a)  The  geometry  of  the  domain  Dq  (b)  The  attracting  set  of  fixed  points  Mo; 

each  point  p  in  Mq  has  a  n-dimensional  stable  manifold  /0S  (p)  (unperturbed 

stable  fiber  at  p)  satisfying  (x,  </>)  =  const .  124 

6-2  (a)  The  geometry  of  the  slow  manifold  Me  (b)  A  trajectory  intersecting  a 

stable  fiber  /es (p)  converges  to  the  trajectory  through  the  fiber  base  point  p.  124 
6-3  Sudden  changes  in  the  velocity  field  delay  convergence  to  the  slow  manifold.  126 
6-4  Particle  velocity  (blue  solid  curve)  attracted  by  a  concentrated  layer  of  prob¬ 
ability,  a  stochastic  slow  manifold.  The  red  solid  curve  denotes  the  mean 
of  the  manifold  and  the  dotted  curves  define  the  local  spread  of  probability 

around  the  mean .  136 

6-5  Clustering  manifold  is  independent  from  the  flow  direction .  141 

6-6  Stochastic  double  gyre  for  the  illustration  of  the  inertial  particles  motion; 

Re  =  25  (See  chapter  5  for  details) .  143 


13 


6-7  Mean  value  of  the  stochastic  slow  manifold  governing  the  motion  of  inertial 


particles  for  the  double  gyre  stochastic  flow .  144 

6-8  Variance  of  the  stochastic  slow  manifold  descrbing  of  motion  inertial  particles 

for  t  =  100.0  and  t  =  100.5 .  145 

6-9  Variance  of  the  stochastic  slow  manifold  descrbing  of  motion  inertial  particles 

for  t  =  101.0  and  t  =  101.5 .  146 

6-10  (a)  Rapid  convergence  to  the  stochastic  slow  manifold  during  the  initial  phase 
of  motion.  The  colormap  denotes  the  local  variance  of  the  slow  manifold 
as  (x,  t)  .  The  vertical  coordinate  shows  the  distance  of  the  stochastic  dy¬ 
namics  from  the  mean  slow  manifold,  (b)  x—  component  of  the  particle 


velocity,  resolved  according  to  Maxey-Riley  equation  for  a  particular  flow 


realization  (blue  solid  curve).  The  red  lines  indicate  the  local  spread  of  prob¬ 
ability  around  the  mean  slow  manifold  at  the  particle’s  location,  (c)  same  as 

(b)  but  now  the  distance  from  the  slow  manifold  is  shown .  148 

6-11  Same  as  6-10  but  for  a  later  time  instant .  149 

6-12  Comparison  of  stochastic  inertial  equation  (6.40)  and  direct  Monte-Carlo 

simulation  for  t  =  100, 101, 102 .  151 

6-13  Same  as  Figure  6-12  for  t  =  103,  104 .  152 

6-14  Concentration  field  for  finite  size  particles  (heavy  particles)  for  two  different 
time  instants.  The  blue  lines  indicate  streamlines  for  the  mean  flow.  The  red 
lines  indicate  clustering  manifolds  for  the  stochastic  flow  field .  153 

B-l  Computational  algorithm  for  the  adaptive  DO  equations .  169 


C-l  Trajectories  jump  away  from  M(t)  over  the  unstable  subset  Mu(t),  but  re¬ 
turn  to  M(t)  over  the  stable  subset  Ms(t).  The  figure  assumes  that  M(t) 
is  a  normally  hyperbolic  attracting  manifold,  in  which  case  the  jumping  tra¬ 
jectory  keeps  approaching  the  same  underlying  trajectory  on  M(t)  by  the 

invariant  foliation  of  the  stable  manifold  Ws(M(t )) .  173 

C-2  The  operator  nj+* ,  |  NpM{t)  maps  vectors  in  the  normal  space  NpM ( t ) 

to  vectors  in  the  normal  space  Npt+s^M (t  +  s).  The  NILE  a(jr,  t)  is  the  ex¬ 
ponential  rate  at  which  the  norm  of  the  above  operator  grows  in  the  limit  of 
infinitesimally  small  s.  Therefore,  a(jp\  t )  measures  the  exponential  rate  at 
which  the  normal  component  of  vectors  normal  to  the  manifold  M(t)  grows 
over  very  short  times.  (The  time  r  is  arbitrary  within  the  interval  [t,  t  +  s].)  174 

C-3  The  manifold  M (t)  as  a  local  graph  over  the  x  variables .  175 


14 


List  of  Tables 


5.1  Reference  values  of  parameters  in  the  barotropic  QG  model  (dimensional).  .  98 

5.2  Reference  values  of  parameters  in  the  barotropic  QG  model  (non-dimensional).  98 

6.1  Reference  values  of  parameters  in  the  barotropic  QG  model  used  for  the  study 

of  finite-size  particles  (dimensional) .  143 


15 


16 


Chapter  1 

Introduction 


Dynamical  systems  play  a  central  role  in  applications  of  mathematics  to  natural  and  en¬ 
gineering  sciences.  However,  dynamical  systems  governing  real  processes  always  contain 
some  elements  characterized  by  uncertainty  or  stochasticity.  Uncertainties  may  arise  in  the 
system  parameters,  the  boundary  and  initial  conditions,  and  also  in  the  ‘external  forcing’ 
processes.  Also,  many  problems  are  treated  through  the  stochastic  framework  due  to  the 
incomplete  or  partial  understanding  of  the  governing  physical  laws.  In  all  of  the  above  cases 
the  existence  of  random  perturbations,  combined  with  the  complex  dynamical  mechanisms 
of  the  system  itself  can  often  lead  to  a  rapid  growth  of  the  uncertainty  in  the  dynamics 
and  state  of  the  system.  Such  rapid  growth  can  distribute  the  uncertainties  to  a  broadband 
spectrum  of  scales  both  in  space  and  time,  making  the  system  state  particularly  complex. 

In  the  past  decades  an  increasing  number  of  problems  in  continuum  theory  have  been 
treated  using  stochastic  dynamical  theories.  Such  problems  are  mainly  described  by  stochas¬ 
tic  partial  differential  equations  (SPDEs)  and  they  arise  in  a  number  of  areas  including  fluid 
mechanics,  elasticity,  and  wave  theory,  describing  phenomena  such  as  turbulence,  random 
vibrations,  flow  through  porous  media,  and  wave  propagation  through  random  media.  This 
is  but  a  partial  listing  of  applications  and  it  is  clear  that  almost  any  phenomenon  described 
by  a  field  equation  has  an  important  subclass  of  problems  that  may  profitably  be  treated 
from  a  stochastic  point  of  view. 

Probably  the  most  characteristic  representative  from  this  family  of  problems  are  turbu¬ 
lent  flows.  In  turbulence  the  spatial  and  temporal  dependence  of  the  instantaneous  values 
of  the  fluid  dynamics  fields  have  a  very  complex  nature.  Moreover,  if  turbulent  flow  is  setup 
repeatedly  under  the  same  conditions,  the  exact  values  of  these  fields  will  be  different  each 
time.  However,  even  though  the  details  of  the  flow  maybe  different  over  various  runs,  it 
has  been  observed  that  their  statistical  properties  remain  similar,  or  at  least  coherent  over 
certain  finite-time  and  space  scales.  These  observations  lead  to  the  natural  conjecture  that 
statistical  modeling  or  statistical  averaging  over  appropriate  spatial  and  temporal  scales 
maybe  more  efficient  for  the  description  of  these  phenomena. 

In  this  work,  we  develop  a  new  methodology  for  the  representation  and  evolution  of 
the  complete  probabilistic  response  of  infinite-dimensional,  random,  dynamical  systems. 
More  specifically,  we  derive  an  exact,  closed  set  of  evolution  equations  for  general  nonlinear 
continuous  stochastic  fields  described  by  a  Stochastic  Partial  Differential  Equation  (SPDE). 
By  hypothetizing  a  decomposition  of  the  solution  field  into  a  mean  and  stochastic  dynamical 
component  that  are  both  time  and  space  dependent,  we  derive  a  system  of  field  equations 
consisting  of  a  Partial  Differential  Equation  (PDE)  for  the  mean  field,  a  family  of  PDEs  for 
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the  orthonormal  basis  that  describe  the  stochastic  subspace  where  the  stochasticity  ‘lives’ 
as  well  as  a  system  of  Stochastic  Differential  Equations  that  defines  how  the  stochasticity 
evolves  in  the  time  varying  stochastic  subspace.  These  new  Dynamically  Orthogonal  (DO) 
evolution  equations  are  derived  directly  from  the  original  SPDE,  using  nothing  more  than 
a  dynamically  orthogonal  condition  on  the  representation  of  the  solution.  This  condition 
is  the  ‘key’  to  overcome  the  redundancy  issues  of  the  full  representation  used  while  it  does 
not  restrict  its  generic  features.  Therefore,  we  do  not  assume  an  a  priori  representation 
neither  for  the  stochastic  coefficients,  nor  for  the  spatial  structure  of  the  solution;  all  this 
information  is  obtained  directly  by  the  system  equations,  boundary  and  initial  conditions. 
If  additional  restrictions  are  assumed  on  the  form  of  the  representation,  we  recover  both 
the  Proper-Orthogonal-Decomposition  (POD)  equations  and  the  generalized  Polynomial- 
Chaos  (PC)  equations.  For  the  efficient  treatment  of  the  strongly  transient  character  on 
the  systems  described  above,  we  derive  adaptive  criteria  for  the  variation  of  the  stochastic 
dimensionality  that  characterizes  the  system  response.  Those  criteria  follow  directly  from 
the  dynamical  equations  describing  the  system.  We  also  describe  how  information  obtained 
from  full-field  data  inputs  can  be  merged  with  the  numerically  evolved  stochastic  fields 
within  the  context  of  DO  equations. 

Since  the  basis  of  the  stochastic  subspace  is  evolving  according  to  the  system  SPDE, 
fewer  modes  are  needed  to  capture  most  of  the  stochastic  energy  relative  to  the  classic  POD 
method  that  fixes  the  form  of  the  basis  a  priori,  especially  for  the  case  of  transient  responses. 
On  the  other  hand,  since  the  stochasticity  inside  the  dynamically  varying  stochastic  sub¬ 
space  is  described  by  a  reduced-order,  exact  set  of  SDEs,  we  avoid  the  large  computational 
cost  of  PC  methods  to  capture  non-Gaussian  behavior.  Therefore,  by  allowing  the  stochas¬ 
tic  subspace  to  change  we  obtain  a  better  understanding  of  the  physics  of  the  problem  over 
different  dynamical  regimes  without  having  the  well  known  cost  or  divergence  issues  resulted 
from  irrelevant  representations  of  spatial  (in  POD  method)  or  stochastic  structure  (in  PC 
method).  We  illustrate  and  validate  this  novel  technique  by  solving  the  2D  Navier-Stokes 
equations  in  various  geometries  and  compare  with  direct  Monte  Carlo  simulations.  We  also 
apply  the  derived  framework  for  the  study  of  the  statistical  responses  of  an  idealized  ‘double 
gyre’  model,  which  has  elements  of  ocean,  atmospheric  and  climate  instability  behaviors. 

Finally,  we  use  our  new  stochastic  description  for  flow  fields  to  study  the  motion  of 
inertial  particles  in  flows  with  uncertainties.  Inertial  or  finite-size  particles  in  fluid  flows 
are  commonly  encountered  in  nature  (e.g.,  contaminant  dispersion  in  the  atmosphere)  as 
well  as  in  technological  applications  (e.g.,  chemical  systems  involving  particulate  reactant 
mixing)  and  as  it  has  been  observed  both  numerically  and  experimentally,  their  dynamics 
can  differ  markedly  from  infinitesimal  particle  dynamics.  Here  we  use  recent  results  from 
stochastic  singular  perturbation  theory  in  combination  with  the  DO  representation  of  the 
random  flow,  in  order  to  derive  a  reduced  order  inertial  equation  that  will  describe  efficiently 
the  stochastic  dynamics  of  inertial  particles  in  arbitrary  random  flows. 


1.1  Preview  of  chapters 

This  thesis  is  organized  as  follows.  In  Chapter  2  we  give  the  necessary  notation  and  sum¬ 
marize  basic  properties  for  random  fields.  We  present  in  detail  a  generalized  form  of  the 
Karhunen  Loeve  expansion,  which  is  a  fundamental  tool  for  the  reduction  of  the  stochastic 
dimensionality  that  characterizes  a  given  problem.  We  also  present  a  geometrical  interpre¬ 
tation  of  its  properties  and  we  illustrate  how  it  can  be  used  for  the  efficient  time  and  space 
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dependent  representation  of  random  fields  that  characterize  fluid  motion  and  have  a  priori 
properties  such  as  non-divergence. 

Chapter  3  is  the  central  theoretical  part  of  the  thesis.  Here  we  derive  an  exact,  closed 
set  of  evolution  equations  for  general  continuous  stochastic  fields  described  by  a  Stochastic 
Partial  Differential  Equation  (SPDE).  The  derivation  is  based  on  a  novel  condition,  the 
dynamical  orthogonality,  on  the  representation  of  the  solution,  which  as  we  prove,  it  comes 
naturally  without  imposing  any  constraints  on  the  form  of  the  response.  Based  on  this 
condition  we  derive  a  system  of  field  equations  consisting  of  a  Partial  Differential  Equa¬ 
tion  (PDE)  for  the  mean  field,  a  family  of  PDEs  for  the  orthonormal  basis  that  describe 
the  stochastic  subspace  as  well  as  a  system  of  Stochastic  Differential  Equations  that  de¬ 
fines  how  the  stochasticity  evolves  in  the  time  varying  stochastic  subspace.  We  also  prove 
that  under  additional  restrictions  on  the  form  of  the  representation,  the  DO  field  equa¬ 
tions  reproduce  both  the  Proper-Orthogonal-Decomposition  equations  and  the  generalized 
Polynomial-Chaos  equations;  thus  the  new  methodology  unifies  these  two  approaches. 

The  scope  of  Chapter  4  is  to  develop  adaptive  criteria  for  the  dimensionality  of  the 
stochastic  subspace  in  the  context  of  the  dynamically  orthogonal  field  equations.  We  present 
adaptive  criteria  for  the  contraction  and  expansion  of  the  stochastic  subspace  and  we  also 
illustrate  how  the  new  stochastic  dimensions  should  be  chosen  (when  the  stochastic  subspace 
should  be  expanded)  according  to  stability  arguments.  These  criteria  are  based  on  the 
current  stochastic  response  of  the  system  and  they  use  a  priori  hypotheses  on  the  spectrum 
of  the  orthogonal  complement  of  the  stochastic  subspace.  Note  that  we  restrict  ourselves 
to  the  ‘internal’  adaptation,  i.e.  we  assume  that  the  realizations  of  the  full  fields  resulting 
from  our  simulations  are  the  new  information  used  in  the  adaptation.  This  is  different  from 
adaptation  to  external  irregular  data. 

In  Chapter  5  we  apply  the  Dynamically  Orthogonal  field  equations  to  the  case  of  two 
dimensional  random  flows  described  by  Navier-Stokes  equations  with  and  without  the  Cori¬ 
olis  force  due  to  a  rotating  reference  frame.  In  the  first  two  sections  we  formulate  the 
problem  and  we  derive  closed,  evolution  equations  for  the  mean  field,  the  scalar  stochastic 
coefficients,  and  the  DO  modes.  We  also  discuss  the  case  of  stochastic  boundary  conditions 
and  we  prove  that  this  family  of  problems  can  always  be  reformulated  as  problems  with 
deterministic  boundary  conditions  and  suitable  forcing.  Subsequently,  we  present  numerical 
results  for  specific  geometries  and  forcing  configurations  and  we  will  examine  convergence 
properties  of  the  proposed  methodology.  In  the  last  section  of  the  chapter  we  consider  an 
idealized  model  for  the  description  of  the  temporal  variability  of  the  wind-driven,  vertically 
averaged,  ocean  circulation.  The  aim  of  this  section  is  to  study  the  stochastic  response  of 
this  model  for  different  forcing  parameters  and  Reynolds  regimes.  Through  the  developed 
stochastic  framework  we  shall  prove  that  in  the  unstable  regimes,  as  those  are  predicted  by 
the  deterministic  theory,  the  system  converges  to  a  stochastic  steady  state  response  which 
is  characterized  by  finite  variance  that  is  smaller  than  the  energy  of  the  mean  flow.  For 
larger  Reynolds  or  forcing  amplitude,  we  find  that  this  variance  may  become  comparable 
with  the  energy  of  the  mean  flow  giving  rise  to  periodic  or  even  chaotic  responses. 

In  Chapter  6  we  apply  our  theoretical  and  numerical  results  derived  for  the  description 
of  stochastic  flows  in  order  to  study  the  motion  of  finite-size  particles  in  flows  with  uncer¬ 
tainty.  Specifically,  we  examine  the  coupled  effects  due  to  inertia  and  flow  stochasticity. 
In  the  first  part  of  the  chapter  we  summarize  theoretical  results  for  particles  in  determin¬ 
istic  flows.  Subsequently  we  present  results  for  the  stochastic  case.  Specifically,  we  prove 
that  the  velocity  of  finite-size  particles  is  governed  by  a  stochastic  slow  manifold,  a  ‘layer’ 
of  probability  around  the  deterministic  slow  manifold  derived  previously  for  deterministic 
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flows.  Based  on  a  stochastic  reduction  on  this  manifold  we  derive  a  stochastic  inertial 
equation  that  governs  the  motion  of  particles  and  which  includes  new  terms  expressing  the 
coupled  effect  of  particles  inertia  and  stochasticity.  In  the  second  part  of  the  chapter  we  first 
illustrate  numerically  the  convergence  of  the  particles  stochastic  velocity  to  the  stochastic 
slow  manifold.  We  validate  the  derived  inertial  equation  for  a  specific  example  and  we 
analyze  the  coupled  effects  of  particles  inertia  and  flow  stochasticity  on  the  preferential 
concentration  of  particles.  In  Chapter  7  we  summarize  the  contributions  of  this  work. 
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Chapter  2 

Representations  of  stochastic  fields 


Abstract 

The  primary  purpose  of  this  chapter  is  to  give  the  necessary  notation  and  summarize  basic 
properties  for  random  fields.  In  the  first  section  we  give  the  definition  of  a  stochastic  phe¬ 
nomenon.  Subsequently  we  provide  with  a  brief  description  of  the  mathematical  tools  used 
for  the  representation  of  stochastic  fields  such  as  probability  density  functions,  moments, 
and  the  characteristic  functional.  We  also  prove  that  those  different  descriptions  are  equiv¬ 
alent.  The  next  section  refers  to  Karhunen  Loeve  expansion,  which  is  a  fundamental  tool 
for  the  reduction  of  the  stochastic  dimensionality  that  characterizes  a  given  problem.  We 
give  a  geometrical  interpretation  of  its  properties  and  we  illustrate  how  it  can  be  used  for 
the  efficient  representation  of  random  fields  that  characterize  fluid  motion  and  have  a  priori 
properties  such  non-divergencess. 

2.1  Introduction 

Many  problems  arising  in  nature  and  technology  can  be  profitably  treated  from  a  stochastic 
point  of  view.  A  common  characteristic  for  these  problems  is  the  presence  of  uncertainties  in 
the  system  parameters,  and  disordered  or  random  perturbations  in  the  dynamical  variables 
that  describe  the  system  state.  Also,  many  problems  are  treated  through  the  stochastic 
framework  due  to  the  incomplete  or  partial  understanding  of  the  governing  physical  laws  or 
simply  because  the  stochastic  approach  is  the  most  efficient  way  of  doing  computations  (e.g. 
turbulence).  In  all  of  the  above  cases  the  existence  of  random  perturbations,  combined  with 
the  nonlinearities  of  the  system  often  leads  to  their  rapid  growth  which  causes  distribution 
of  energy  to  a  broadband  spectrum  of  scales  both  in  space  and  time,  making  the  system 
state  particularly  complex. 

Probably  the  most  characteristic  representative  from  this  family  of  problems  are  turbu¬ 
lent  flows.  In  turbulence  the  spatial  and  temporal  dependence  of  the  instantaneous  values 
of  the  fluid  dynamics  fields  have  a  very  complex  nature.  Moreover,  if  turbulent  flow  is 
setup  repeatedly  under  the  same  conditions,  the  exact  values  of  these  fields  will  be  different 
each  time  (see  e.g.  Monin  and  Yaglorn  [102]).  However,  even  though  the  details  of  the 
flow  maybe  different  over  various  runs,  it  has  been  observed  that  their  statistical  proper¬ 
ties  remain  similar  leading  to  the  natural  conjecture  that  statistical  modeling  or  statistical 
averaging  over  appropriate  spatial  and  temporal  scales  (‘spectral  windows’,  [105])  maybe 
more  efficient  for  the  description  of  these  phenomena. 

The  primary  purpose  of  this  chapter  is  to  introduce  the  mathematical  tools  used  for  the 
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description  of  random  fields.  Before  we  proceed  to  the  definition  of  a  random  or  stochastic 
fields  we  shall  first  give  a  more  formal  definition  of  what  we  mean  by  a  random  or  stochastic 
phenomenon.  We  consider  a  sequence  of  runs  for  the  same  experiment  IP  (where  oj  denotes 
an  arbitrary  realization)  that  describe  a  given  phenomenon,  IP1,  IP2,...,  IPn,...and  the 
corresponding  sequence  of  results  iP1,  iP2, ...,  RU)n, ....  In  many  cases  the  following  relation 
holds  for  the  outcomes  of  the  various  runs 

d  for  all  i,j  (2.1) 

where  d  is  a  suitable  metric  (i.e.  distance,  see  e.g.  [104])  and  is  a  predefined  level  of 
tolerance.  The  above  behavior  leads  to  the  commonly  expected  result  that  different  runs  of 
the  same  experiment  leads  to  close  results.  From  the  above  discussion  we  have  the  following 
definition  [7] 

Definition  1  A  phenomenon  will  be  called  deterministic  if,  for  every  experiment  IP  that  is 
associated  with  it,  any  sequence  of  experimental  runs  IP'  satisfies  eq.  (2.1)  with  sufficiently 
small  ew.  In  contrary  if  relation  (2.1)  is  not  valid  for  one  or  more  experiments  then  the 
phenomenon  will  be  called  nondeterministic. 

Therefore,  the  definition  of  a  deterministic  or  nondeterminisitc  phenomenon  depends 
strongly  on  the  tolerance  level  ew  as  well  as  on  the  metric  d.  Even  though  an  experiment 
associated  with  a  nondeterministic  phenomenon  presents  important  variability  in  terms  of 
results,  this  picture  may  change  if  we  consider  mean  values  of  the  outcomes  for  a  large 
sequence  of  runs,  e.g.  the  ensemble  mean 

— -  RU1  +  IP2  +  ...  +  RPN 


Then,  it  is  possible  to  have  sufficiently  close  mean  values  Rf)  over  different  sequences  of 
runs  {IP}-  for  the  same  experiment  IP 

d  (^R%. ,  R%.  ^  <  ew  for  any  large  experimental  sequences  {lP}j ,  {IP}^ 

In  this  case  the  experiment  is  said  to  present  statistical  stability  [163].  Using  this  concept 
we  proceed  to  the  definition  of  a  stochastic  or  random  phenomenon  ([163],  [7]) 

Definition  2  A  nondeterministic  phenomenon  will  be  called  stochastic  or  random  if  the 
results  of  any  associated  experiment  with  it  IP,  present  statistical  stability. 

We  shall  now  define  the  appropriate  tools  for  the  mathematical  modeling  of  stochastic 
phenomena.  In  the  theoretical  modeling  of  random  phenomena  the  basic  role  is  played  by 
the  probability  space  (£l,£n,V)  .  The  set  U,  which  we  call  sample  space,  includes  all  the 
possible  outcomes  of  a  given  phenomenon.  £q  is  a  a— algebra  of  subsets  of  II  [67],  i.e.  a  set 
of  subsets  that  has  the  following  properties  i)  H  6  £q,  ii)  if  A  £  £^  then  its  complement  is 
also  in  £q,  iii)  Every  countable  union  of  the  elements  of  £q,  i.e.  every  At  £  £q,  is  in  £q. 
Based  on  these  two  concepts  we  define  the  probability  measure  Pq  as  a  set  function  from 
£q  to  the  interval  [0, 1]  which  satisfies  the  three  axioms  of  probability  ([67],  [135]). 

For  every  physical  realization  w  6  (1,  we  may  correspond  a  quantity  u  {to)  £  IA ,  where 
U  is  a  suitable  set  (e.g.  a  subset  of  M,  a  set  of  functions,  or  a  set  of  fields).  Therefore 
the  abstract  set  II  contains  all  the  possible  physical  realizations  of  the  phenomenon,  while 


22 


1A  contains  their  corresponding  mathematical  description.  For  example,  in  the  case  of  a 
random  flow,  wed  will  represent  a  particular  physical  realization  of  the  flow,  while  u  (uj) 
will  represent  a  specific  mathematical  quantity  associated  with  the  flow  e.g.  the  velocity 
field.  This  leads  to  the  concept  of  a  stochastic  process  or  stochastic  field.  Specifically, we 
have  the  following  formal  definition 


Definition  3  A  stochastic  field  is  a  function  u(x,  t;uj)  =  {uj  (x,t\  w)}jL1  £  x  G 
D  C  Mn,f  G  T,  uj  G  ft,  defined  on  a  sample  space  17,  a  spatial  domain  D ,  and  a  time 
interval  T,  such  that,  for  every  x  G  D  C  Mn,t  G  T  and  every  real  vector  r  G  Mm  the  set 
{uj  :  uj  (. x,t\u )  <  rj,  j  =  1,  ...,m}  is  inside  £q. 


Having  a  stochastic  field,  there  are  at  least  three  different  kinds  of  methods  to  obtain 
statistically  averaged  properties.  They  are  space  averages,  time  averages,  and  ensemble 
averages.  The  usefulness  of  space  averages  is  limited  to  fields  that  are  statistically  homoge¬ 
neous  or  at  least  approximately  homogeneous  over  scales  larger  than  those  of  the  random 
features.  Similarly,  time  averages  are  useful  only  if  the  stochastic  field  is  in  effect  statisti¬ 
cally  stationary  over  time  scales  much  larger  than  the  time  scale  of  the  stochastic  properties. 
In  both  of  the  above  cases  averaging  over  the  appropriate  ‘spectral  window’  (i.e.  range  of 
spatial  and  temporal  scales,  [105])  may  significantly  simplify  the  problem.  The  third  type 
of  average,  ensemble  averages,  does  not  assume  anything  on  the  statistical  characteristics 
of  the  random  field  (e.g.  homogeneity  or  stationarity)  and  for  this  reason  it  is  the  most 
versatile.  In  what  follows  every  averaged  quantity  will  be  in  the  sense  of  ensemble  average 
and  will  be  denoted  as 


E“[uH] 


..  u(u>l)  +  u(uj2)  +  ...  +  u(u>n) 

hm  - ■ — — - 

W— >  oo  N 


where  Ui  G  fl  are  specific  realizations.  In  the  following  sections  we  will  present  the  essential 
tools  used  for  the  analysis  and  description  of  stochastic  fields  in  terms  of  their  statistical 
and  physical  properties. 


2.2  The  family  of  probability  density  functions 

In  this  section  we  will  give  the  definitions  of  probability  density  functions  and  also  present 
the  essential  notation  that  will  be  used  in  the  rest  of  the  thesis.  Given  a  stochastic  vector 
field  u  (x,  f;  uj)  defined  as  above,  we  have  for  every  collection  of  spatial  positions  xi,  X2, ...,  xjv 
and  time  instants  t\,t2,  ...,tjv  the  joint  probability  distribution  function 

Fu(xi,t i),...,u(xjv,tjv)  (ui>  UN )  =  V[{uj:u  (xi,  H;  u)  <  ui  n  ...  n  u  (xjv,  tN\  uj)  <  Uat}] 

where  each  inequality  is  meant  in  the  component- wise  sense,  i.e.  Uj  (aq ,t\\u)  <  u\j ,  j  = 
1  This  family  (the  term  ’family;  follows  from  the  fact  that  the  joint  distribution 

function  is  defined  for  arbitrary  combinations  of  time  instants  and  space  locations)  will 
always  satisfy  the  Kolmogorov  compatibility  conditions  [67] 

1)  the  symmetry  condition:  if  {ii,  Z2, in}  is  a  permutation  of  numbers  1,2, ...,  N  then 
for  arbitrary  N  >  1 

^u(xi,ti),...,u(xiv,tjv)  (uii  •••>  un)  =  ^(x11,tn),...,u(xiJV,tlJV)  (UU)  •••)  u*at)  • 
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2)  the  consistency  condition:  for  M  <  N  we  have 

uj*5oo  -^u(xl,tl),...,U (xM,tM),  —  u(x/IC,tjv)  (Ul>  UN )  —  -^u(xi,tl),...,u(xM,tM)  (Ul>  UA^)  • 

j>M 

The  above  family  of  probability  distribution  functions  defines  uniquely  the  stochastic  field 
u  (x,  t;  (o)  .  Therefore  a  complete  statistical  characterization  of  a  random  field  involves  the 
characterization  of  the  full  family  of  probability  distribution  functions. 

In  the  special  case  where  i?1u(xi)t1) ^.]U(XlVitjv)  (ui,  ...,u/v)  is  differentiable  with  respect  to 
its  arguments  (ui,  ...,ujv)  for  every  collection  of  spatial  locations  and  time  instants  we  can 
alternatively  use  the  probability  density  function  (see  e.g.  [109]) 

dN 

f u(xi,ti),...,u(xjv ,tjv)  (Ul>  •”>  U-W)  —  -^u(xi,ti),...,u(xjv,tjv)  (Ul’  Un) 

where 

d ■  _  dm ■ 

du  du\dll2  •  •  •  dum 

Note  that  for  the  special  case  of  scalar  fields  described  by  the  probability  distribution 
function 

Fu(*.1,t1),...,u(x.N,tN)  (Uu  -,uN)  =  V[{uj  :  u  (x1;  ti;  cu)  <  ui  n  ...  n  u  (xN,  tN\  w)  <  uat}]  , 
the  definition  of  the  probability  density  function  has  the  simpler  form 

dN 

/ii(xi,ti),...,u(xjv,tjv)  (Ul>  —>un)  —  QUn  Fu(xi,ti),...,u(xN,tN)  (wl)  UN )  • 

In  the  general  case  the  family  of  probability  density  functions  satisfies  the  following  condi¬ 
tions 

1)  the  non-negativity  condition 

/u(xi,ti),...,u(xjv,tjv)  (uli  U N )  ^  0 

2)  the  normalization  property 

j  /u(xi,ti),...,u(xjv,tjv)  (Ul,  ...,  Ujv)  dui...du7v  1 

Rjv.m 

3)  the  marginal  property 

j  /u(xi ,t\ ),..., u(xjv,tjv)  (^1  >  •  Ujv)  1  •  •  /u(xi ,ti u(xmi4m)  (^1  >  •••  >  ^ M ) 

R(Af -M).m 

For  the  fields  that  we  will  consider  in  the  present  work  we  will  assume  that  the  probability 
distribution  functions  of  every  order  (i.e.  for  any  number  of  joint  time  instants  and  space 
locations  -  e.g.  second  order  are  the  joint  probability  density  functions  involving  two  time 
instants  and  space  locations)  exist  and  they  are  also  differentiable.  In  this  case  an  alternative 
tool  for  the  description  of  a  stochastic  field  is  given  by  the  family  of  characteristic  functions 
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defined  as 


</>u(xi,ti),...,u(xjv,tAr)  (£l’  "■!  6v)  -F  [/u(xi,ti),...,u(xjv,tjv)  ('?  ’). 


(  N  T  } 

Eu 

exp  | 

\  Fi  J 

where  T  denotes  the  Fourier  transform.  The  family  of  characteristic  functions  possess  the 
following  properties 

1)  the  normalization  property 

</)u(xi,ti),...,u(xjv,ijv)  (^’  ^ 

2)  the  boundness  property 

|</)u(xi,ti),...,u(xJv,tiv)  (Cl)  •••i6v)|  <  1  for  all  (£i,...,£jv)  G  M.N'm 

3)  the  symmetry  property 

<^u(xi,ti),...,u(xw,tjV)  (£li  •••)  £,n)  —  </>u(xi,ti),...,u(xj v,*jv) 

where  *  denotes  the  complex-conjucate. 

4)  the  marginal  property 

</)u(xi ,ti),...,u(xiv,tjv)  (?!)••■)  ;  0,  ,  0)  0u(xi,ti u(xm 4m)  (£l  >  ■  •  )  • 

5)  positive  definiteness:  for  every  vector  c  G  ZM  (with  c*  denoting  the  complex  conjugate 
of  c)  we  have 


M  M 

^  ^^>u(xi4i),...,u(xjv,tjy)  (£l  i  £l ji ■  ■  •  ■  3  £,Mi  £,Mj)  CiCj  T  0 

J=1  *=1 

for  any  collection  of  vectors  Mm  3  £ij,  i,  j  =  1, ...,  A/. 


2.3  The  moment  system 


The  next  important  concept  for  the  description  of  stochastic  or  random  fields  is  the  statisti¬ 
cal  moments.  For  a  stochastic  vector  field  u  (x,  t;  uj)  defined  as  above,  we  have  the  statistical 
moments  of  the  first  order 

u  (x,  t)  =  E?  [u  (x,  t;  w)]  =  j u/u(Xit)  (u)  du  G  Mm 

R 
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It  is  often  necessary  to  investigate  the  joint  behavior  of  the  field  at  two  different  spatiotem- 
poral  locations.  For  this  case  we  define  the  correlation  operator  (or  second  moment) 


R-u(xi,ti;u))u(x2,i2;w)  ^ 


u(xi,ti;u;)  u(x2,f2;w)J 


j  U^ll2  /u(xi,ii)u(x2,t2)  (^1)  U2)  dll  1  (IU'2  G  M 
lm2 


An  alternative  definition  is  given  by  the  correlation  coefficient  which  is  a  normalized  version 
of  the  correlation  operator. 

In  many  occasions  it  is  often  more  convenient  to  work  with  central  moments.  The  second 
order  central  moments  are  described  by  the  covariance  operator  (or  central  second  moment) 


C1  = 

^u(xi,ti;cj)u(x2,t2;^)  ^ 


(u(xi,ti;w)  -  u(xi,fi;w))  (u  (x2,t2;u;)  -  u  (x2,  t2;  uj))j 


Since,  for  what  will  follow  we  will  make  extensive  use  of  the  above  operator  at  the  same 
time  instants  t±  =  £2  but  at  different  locations  we  will  use  the  following  notation  for  the 
spatial  covariance 


^ u(- ,t;u))u(- (^>  Y )  ^-'u(x,t;u;),u(y,t;aj) 


=EU 


(u  (x,  t;  uj)  -  u  (x,  t;  uj))  (u  (y,  t;  uj)  -  u  (y ,  i;  cu))J 


The  above  quantities  define  the  second-order  statistical  characteristics  for  the  field  u  (x,  f;  uj). 
Their  knowledge  is  sufficient  for  a  complete  stochastic  description  for  the  case  of  Gaussian 
fields  (see  e.g.  [109]).  However,  for  the  general  case,  the  complete  family  of  moments  is 
required.  Specifically,  a  complete  stochastic  description  of  u  (x,  f;  uj)  requires  the  knowledge 
of  all  moments  of  order  r 


E“  [u 


n 


J 


fljm.r 


(xi,H;  uj)  uj2  (x2,t2;w)  ...ujr  (x,r,fr;cu)] 

uj  1  Hj2  ■  •  *  ujr  /u(xi  ,ti)u(x2,i2),...,u(xI.,tr)  (ui,  U2  j  ••■5  Ur)  dll  ]  dll2  • .  .dllr 


for  every  combination  of  spatial  locations  xi,  x2, ...,  xr,  time  instants  t\,  i2,  ■  tr,  indices 
j  =  (j\ ,  j2,  ...,jV) ,  (with  jk  G  {1,  ...,m})  and  r  =  1,2, ...  .  Note  that  in  the  above  definition 
Uj1  denotes  the  jj  component  of  ui  and  so  on. 

2.3.1  Connection  with  the  family  of  characteristic  functions 

We  saw  previously  that  the  system  of  statistical  moments  is  connected  to  the  family  of 
probability  density  functions.  This  is  also  true  for  the  family  of  characteristic  functions. 
Specifically,  we  can  express  any  statistical  moment  as  (see  e.g.  [136]) 

Eu  [uh  (xi,H;  uj)  uj2  (x2,i2;  uj)  ...ujr  (xr,tr;uj)] 

1  dr 

=  Ed^~d^~di~  ^u(-idi 

where  £1  ji  is  the  j\  component  of  the  vector  C  and  so  on.  The  above  property  is  a  direct 
consequence  of  the  definition  of  the  characteristic  function  through  the  Fourier  transform 
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of  the  corresponding  density.  Hence  we  have  presented  three  different  approaches  for  the 
complete  probabilistic  description  of  a  stochastic  field  and  we  have  also  recalled  their  con¬ 
nection.  For  all  the  presented  approaches  the  definition  is  given  in  terms  of  a  system  that 
includes  all  possible  combinations  of  spatial  locations  and  time  instants.  In  the  next  section 
we  describe  an  infinite  dimensional  tool  that  captures  the  complete  probabilistic  information 
in  a  singe  functional. 

2.4  The  characteristic  functional 

We  saw  that  the  tools  presented  so  far  follow  a  bottom-up  approach  in  the  sense  that  we 
define  the  global  statistical  behavior  through  the  definition  of  finite-dimensional  joint  statis¬ 
tical  quantities  such  as  probability  density  functions  at  various  locations  and  time  instants 
or  moments  associated  with  them.  A  top-down  approach  will  involve  the  consideration  of 
an  infinite  dimensional  quantity  that  will  be  able  to  reproduce  all  the  finite-dimensional 
information.  Such  a  quantity  is  the  characteristic  functional  defined  directly  in  terms  of  the 
full  probability  measure  V.  More  specifically,  we  have  the  following  definition  ([136],  [102], 
[120]) 

Definition  4  For  a  stochastic  field  u  (x,  t]Lo)  =  {uj  (x ,  t]  lo)}'j1=1  E  Mm,  x  £  D  C  Mn,f  £ 
T,  u  £  11,  the  characteristic  functional  is  defined  as 


where  6  (x,  t)  £  Mm,  x  £  D  C  Mn,  t  £  T,  is  an  arbitrary  field. 

Therefore,  we  see  that  the  characteristic  functional  is  a  generalization  of  the  family  of 
characteristic  functions  and  therefore  it  has  the  same  properties  i.e. 

1)  the  normalization  property 

$u  [0]  =  1 

2)  the  boundness  property 

|<E>u  [$]|  <  1  for  all  d 

3)  the  symmetry  property 

<hu  [0]  = 

4)  positive  definiteness:  for  every  vector  c  £  rLM  (with  c*  denoting  the  complex  conjugate 
of  c)  we  have 

M  M 

fix  —  Ofi  c,c*  >  0 

j= 1 *=1 

for  arbitrary  fields  9i  i  =  1  As  mentioned  at  the  beginning,  the  characteristic  func¬ 

tional  contains  the  full  probabilistic  information.  In  fact,  through  the  characteristic  func¬ 
tional  we  can  derive  the  complete  system  of  moments,  as  well  as  the  family  of  characteristic 
functions  and  through  them  the  corresponding  family  of  probability  density  functions. 
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2.4.1  Determination  of  the  moments  system 


We  shall  now  illustrate  how  the  characteristic  functional  can  be  used  to  derive  expressions 
for  the  moment  system.  By  considering  the  first  Frechet  derivative  (see  e.g.  [26])  of  the 
characteristic  functional  we  have 


^  _  lim 

69  s 


t,e 


=  iEu 


exp  u  (x,  t]Lo)  [6  (x,  t)  +  s<p  (x,  t)]  dtdhi. 


u  (x,  t;  u)1  <p  (x,  t)  dtchcex p  i  /  /  u  (x,  t;  to)1  9  (x,  t)  dtdx 


s= 0 
T  , 


Considering  the  above  expression  for  6  =  0  we  obtain 


66 


[0;  <p]  =  iEu 


u  (x,  t;  lu)  ip  (x,  t)  dtdx 


Setting  ipj  (x,  t)  =  6  (x  —  xo)  6  (t  —  to)  Sjk  =  6k,Xo,t0  we  obtain 

<5<£>, 
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0;  4,xo.t0]  = iE“  K  (xo,  to;  w)] 


thus 

E“  [«fc  (xo,to;w)]  =  [0;  4,X(),t0]  • 

where  6jk  is  the  Kronecker  delta  and  6  {t  —  to)  is  the  Dirac  delta  function.  Depending  on  the 
smoothness  of  the  characteristic  functional  we  may  consider  higher  order  Frechet  derivatives 
in  order  to  obtain  higher  order  moments.  Specifically,  we  have 


62*u 

69'2 


=  i2Eu 


j  Ju  (x,  f;  uj)T  ipi  (x,  t)  j  Ju  (x,  t;  u)T  p>2  (x,t)dtdx 


from  which  we  obtain  the  correlation  (second  moment)  operator 


_  1  44>u  r  -  -  1 

±t'Ufc(xi,ti;u,)ui(x2,t2;^)—  j2  fiQ2  LU’  °fe,xi,ti > 


We  may  generalize  the  above  procedure  to  obtain  expressions  for  any  moment  characterizing 
the  field  u  (x,  t;  uj) 


Ew  [uh  (x! ,ti;uj)  uj2  (x2,t2;w)  -ujr  (xr,tr;u)j  =  [0;  ^1)Xljtl, 4r,xr,t,] 

From  the  above  equation  we  also  conclude  that  the  existence  of  moments  is  connected  to 
the  smoothness  degree  of  the  characteristic  functional  at  the  origin. 
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2.4.2  Determination  of  the  family  of  characteristic  functions 

Another  important  property  of  the  characteristic  functional  is  its  connection  with  the  family 
of  characteristic  functions.  Specifically,  we  can  generate  any  finite  dimensional  character¬ 
istic  function  simply  by  considering  the  characteristic  functional  for  delta-type  fields,  i.e. 
9k  (x,  t)  =  6  (x  —  xo)  6  (t  —  to)  5jk  =  Sk,x.0,to-  For  one-dinrensional  characteristic  functions 
we  will  have 


$u  [Co^fc,xo,to]  — 


=  Eu 


exp  i£ o 


u 


(x,h  tv)T  6k!X0j0dtdx 


=  Eu  [exp  {i£0Uk  (x0,  t0;  w))] 
=  <Au/s(x0,to)  (£o) 


Using  the  same  approach  we  may  derive  more  complex,  finite-dimensional,  characteristic 
functions 

r  n  i 


(t>Uj1  (xi,t\),...,UjN (xjv,tjv)  (Cl,  £iv)  ‘f'u 


jk  i^k 


lk=l 


where  have  been  defined  in  Section  2.2.  Having  the  full  family  of  finite-dimensional 
characteristic  functions,  we  may  also  obtain  the  corresponding  family  of  probability  density 
functions  by  inverse  Fourier  transform.  Hence  we  have  illustrated  how  the  characteris¬ 
tic  functional  contains  the  full  probabilistic  information  and  it  can  reproduce  any  finite¬ 
dimensional  quantity  connected  with  the  stochastic  field. 


2.4.3  Characteristic  functionals  with  explicit  description 

Although  the  characteristic  functional  provides  a  complete  stochastic  description  for  a  ran¬ 
dom  field  its  main  disadvantage  is  the  lack  of  explicit  formulas  that  can  be  used  for  the 
representation  of  general  stochastic  fields.  In  this  section  we  present  the  characteristic  func¬ 
tional  for  two  special  cases  of  fields:  Gaussian  random  field  and  a  more  generalized  form  of 
characteristic  functional  that  is  capable  of  capturing  more  complex  statistics. 


Gaussian  characteristic  functional 

Gaussian  probability  measures  ([120])  play  a  very  important  role  in  the  theory  of  probability 
in  infinite  dimensional  spaces.  The  major  reason  is  the  existence  of  an  analytic  expression 
for  the  characteristic  functional.  The  technical  reason  behind  this  feature  is  the  property  of 
Gaussian  measures  to  be  defined  completely  through  the  first  and  second  order  statistical 
characteristics.  Below  we  give  the  definition  of  a  Gaussian  characteristic  functional.  We 
first  consider  a  deterministic  field  m  (x,  t)  G  Mm,  x  G  D  C  Rn,f  G  T  that  will  play  the 
role  of  the  mean  field  as  well  as  a  self-adjoint  (symmetric),  positive-definite  operator  with 
finite-trace  (see  e.g.  [104]),  C  (xi,  ti,  X2,  £2)  £  Mmxm,  xi,X2  G  D  C  Mn,ti,f2  €  T  that  will 
play  the  role  of  the  covariance  operator.  Then  we  have  the  following  result  (see  e.g.  [120]) 

Theorem  5  The  characteristic  functional  associated  with  a  normally  distributed  random 
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field  u  (x,  t;  u)  having  mean  m  (x,  i;  uj)  and  covariance  operator  C  (xi,  ti,  X2,  t2)  is  given  by 


log  &u[0]  =  i  J  J  m  (x,  t)1  8  (x,  t)  dtdx 

Rm  T 

-  \  j  Jo  (xi,tl)TC(xi,ti,X2,t2)^(x2,t2)^1^2dxidx2 

i2"‘T2 


Note  that  based  on  the  explicit  expression  for  the  Gaussian  characteristic  functional 
we  may  construct  more  general  representations  of  characteristic  functionals  by  considering 
convex  superpositions  of  different  Gaussian  characteristic  functionals.  For  more  details  we 
refer  to  Sobczyk,  1991  [136]  (p.  21). 


Tatarskii  characteristic  functional 

We  will  now  describe  a  more  general  form  of  characteristic  functional  representation,  first 
introduced  by  Tatarskii,  1995  [142],  This  representation  creates  a  wide  class  of  characteristic 
functionals  and  its  derivation  is  based  on  the  construction  of  a  specific  random  function  (see 
[142]  for  the  description  of  the  random  function).  Here  we  will  state  only  the  representation, 
details  for  the  derivation  can  be  found  in  [142]. 

Theorem  6  Let  a  weight  function  W  (x,  t)  :  D  x  T  — >  [0,oo)  normalized  to  1,  an  arbitrary 
function  g  (xi,  t\,  x2,  f2)  :  D2  x  T2  — >  Mm,  and  a  characteristic  function  <f>v  (-,f)  describing 
a  scalar  stochastic  process.  Then  the  functional 


log  4>u  [9]  = 


J  Jw  (xi,ti)<f)v  |  J  J g  (xi ,  i i ,  x2 itf)T  9  (x2 ,  f2)  dt2dx2 ,  t\ 

Rm  T  T  ) 


dt  i  cix  i 


represents  a  characteristic  functional. 

The  above  functional  has  been  used  successfully  for  the  representation  of  random  wave 
fields  [142]  but  also  for  the  derivation  of  characteristic  functionals  of  probability  measures 
that  are  well  known  measures  in  finite  dimensions,  such  as  the  Gamma  measure,  the  Abel 
measure  and  the  Cauchy  measure  [39].  However,  the  expression  above  may  be  still  too 
restrictive  in  order  to  represent  effectively  stochastic  fields  of  general  form. 


2.5  The  Karhunen  Loeve  expansion 

A  different  approach  for  the  description  of  a  stochastic  field  follows  the  Karhunen-Loeve 
expansion.  This  is  a  linear  decomposition  of  the  stochastic  field  into  deterministic  fields 
multiplied  by  scalar  stochastic  coefficients.  The  deterministic  fields  carry  all  the  spatial 
information  while  the  scalar  coefficients  contain  the  stochastic  information  of  the  response. 
In  this  case  the  description  of  uncertainty  is  done  through  an  expansion  directly  on  the 
state-space  variables  and  not  through  the  joint  probabilities  at  various  spatial  locations  and 
time  instants.  As  we  shall  see  in  the  present  section,  the  Karhunen-Loeve  expansion  has  the 
great  advantage  to  separate  the  probabilistic  structure  of  the  problem  from  the  stochastic 
one.  Moreover,  for  sufficiently  smooth  samples  of  the  stochastic  field,  the  convergence  of 
the  Karhunen-Loeve  series  is  very  rapid.  Specifically,  we  have  the  following  representation 
Theorem  (see  [143],  [59];  see  also  [73],  [81],  [76]  for  oceanic  applications) 
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Theorem  7  An  arbitrary  spatially  mean-square  continuous  random  field  u  (x,  t;u) ,  x  G 
D  C  Mn,f  G  T,  cu  €  fi  can  6e  represented  in  the  form  of  a  series 

OO 

u  (x,  t;ca)  =  u  (x,  t)  +  (t;  u)  u*  (x,  t) 

i= 1 

which  is  mean-square  convergent  for  each  x£  DC  Mn,i  G  T  and 

Yi  (t- uj)  =  J  [u  (x,  t; ta)  -  u  (x,  t)]T  Uj  (x,  t)  dx 

D 

are  zero-mean  stochastic  processes,  mutually  orthogonal,  with  E u  \Yi  (t;  uj)  Yj  ( t ;  a;)]  =  Sij  A?  (i) 
;  where  { u,  (x,  t) ,  A2  (i)}”^  are  i/ie  eigenpairs  associated  with  the  following  eigenvalue  prob¬ 
lem 

J (x,  y)  U*  (y,  t)  dy  =  Af  (t)  u,:  (x,  i) 

D 


Note,  in  the  above  representation  both  the  deterministic  fields  and  the  stochastic  coef¬ 
ficients  are  time-dependent.  Although  many  authors  consider  a  special  case  of  the  above 
representation  where  the  stochastic  coefficients  do  not  evolve  with  time  (see  e.g.  [59],  [134]) 
here  we  will  use  this  time-dependent  representation  in  order  to  obtain  (in  the  next  chapter) 
evolution  equations  for  all  the  quantities  involved.  We  emphasize  that  a  mean-square  con¬ 
tinuous  random  field  u  (x,  t;  u)  has  by  definition  samples  which  are  mean  square  integrable, 

i.e.  the  realizations  of  the  random  field  u  (x,  t;  to)  are  such  that  J u  (x,  t)T  u  (x,  t)  dx  <  oo. 

D 

We  will  denote  the  Hilbert  space  of  spatially  square  integrable  fields  as  L2. 

Note,  that  since  (x,  y)  is  always  self-adjoint  (symmetric)  and  positive 

definite,  we  will  have  a  countable  infinity  of  eigenpairs  with  all  the  eigenvalues  being  real 
positive  and  the  associated  eigenfields  orthogonal  to  each  other.  Also,  limwoc  A?  =  0  [104]. 


2.5.1  A  geometrical  interpretation 

As  we  saw,  Karhunen-Loeve  expansion,  apart  from  the  analytic  requirement  for  square  inte- 
grability,  does  not  impose  restrictions  on  the  stochastic  structure  of  the  random  coefficients, 
i.e.  to  be  necessarily  Gaussian.  If  we  want  to  give  a  more  geometrical  picture  of  its  prop¬ 
erties,  the  Karhunen-Loeve  expansion  finds  the  principal  directions  u,;  (x,  t)  in  the  infinite 
dimensional  space  L2  and  orders  them  so  that  along  the  direction  described  by  Uj  (x,  t)  ,  we 
have  larger  spread  of  probability  relative  to  the  direction  Uj+i  (x,  t),  for  all  i.  Specifically, 
if  we  denote  with 

o-2  [u  (x,  f)]  =  J u  (x,  t,)T  Cu(.jt;u,)u(.)t;a;)  (x,  y )  u  (y ,  t )  dxdy 
D2 
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Figure  2-1:  Ellipsoid  that  contains  the  main  spread  (‘mass’)  of  the  probability  measure; 
defined  by  the  principal  directions  and  the  associated  eigenvalues  of  the  correlation  operator. 


the  spread  or  variance  of  the  probability  measure  associated  with  the  random  field  u  (x,  t;  uj) 
along  the  direction  u  (x,  t)  ,  we  have  the  following  property 

ui  (x,  t )  is  such  that  A2  ( t )  =  a2  [ui  (x,  t)]  =  max  a2  [u  (x,  t )] 

L2 

U2  (x,  t )  is  such  that  A2  ( t )  =  a2  [112  (x,  t)]  =  max  a2  [u  (x,  t )] 

[span{ui}]± 


u,:+i  (x,  t )  is  such  that  A2+1  ( t )  =  a2  [u,;+i  (x,  t )]  =  max  cr2  [u  (x,  t )] 

[span{uj}*.=1] 


where  span{uj}’=|  denotes  the  linear  subspace  spanned  by  the  eigenfields  {u j}*=1  and  -1 
denotes  the  orthogonal  complement.  Therefore,  the  spectral  decomposition  of  the  covariance 
operator  provides  us  with  a  sequence  of  fields  along  which  we  have  a  monotonic  decrease 
on  the  spread  of  the  probability  measure.  Note  that  no  information  about  the  shape  of  the 
probability  measure  is  given  in  this  level  of  analysis  since  the  spread  of  probability  is  the 
maximum  information  that  the  second-order  statistics  can  provide.  Thus,  we  may  think  the 
set  of  eigenpairs  as  an  ellipsoid  in  infinite  dimensions,  with  the  directions  of  the  principal 
axes  defined  by  {uj}^(1  and  the  amplitudes  along  these  axes  defined  by  |A2|  .  This 

ellipsoid  bounds  the  probability  measure,  in  the  sense  that  the  main  spread  (‘mass’)  of  the 
probability  measure  is  contained  in  the  ellipsoid  interior  (Figure  2-1). 

Since,  the  ellipsoid  has  always  a  finite  number  of  axes  which  are  greater  than  an  arbitrary 
constant  e  >  0  (this  is  a  sequence  of  the  property  lirn^oo  A2  =  0)  we  may  argue  that 
Karhunen-Loeve  expansion  gives  a  finite  dimensional  ellipsoid  where  the  main  mass  of 
the  probability  measure  is  contained.  Hence,  we  are  interested  to  characterize  the  detailed 
characteristics  of  the  probability  measure  (such  as  higher  order  moments  or  joint  probability 
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density  functions)  only  along  those  directions  which  are  associated  with  important  spread 
of  probability  or  variance,  i.e.  for  the  part  of  the  probability  measure  contained  in  the 
ellipsoid. 


2.5.2  Application  to  fluid  flows 

In  stochastic  partial  differential  equations  and  especially  in  stochastic  fluid  flows  we  are 
interested  to  describe  and  characterize  random  fields  that  have  given  spatial  properties 
such  as  divergence  free  or  irrotational  fields.  Using  the  Karhunen-Loeve  expansion  we  are 
able  to  represent  very  efficiently  fields  that  have  such  properties.  Moreover,  through  this 
representation  we  shall  prove  properties  which  are  satisfied  by  the  covariance  operator  as 
well  as  higher  order  moments  describing  the  random  field.  Finally,  as  we  shall  illustrate  the 
Karhunen-Loeve  series  provides  us  with  a  very  efficient  tool  to  create  random  samples  of 
fields  with  given  properties  such  as  divergence-free  fields. 

We  first  recall  the  following  fundamental  result  for  flow  fields  [10] 


Theorem  8  Any  spatially  differentiable  flow  field  u  (x,  t)  can  always  be  decomposed  as 

u  (x,  t)  =  u,r  (x,  t )  +  ud  (x,  t)  +  ub  (x,  t) 


where 

i)  ur(x,  t)  is  a  divergence-free  velocity  field  which  can  be  expressed  as 

V.ur  (x,  t)  =  0  ur  (x,  t)  =  V  x  tp  (x,  t ) 

with  if  (x,  t )  being  the  field  streamfunction, 

ii)  u d(x,  t)  is  a  rotational-free  field  which  can  be  expressed  as 

V  x  ud  (x,  t)  =  0  ud  (x,  t)  =  Vq  (x,  t) 


with  A q  (x,  t)  =  div  (x,  t)  being  the  flow  divergence, 

Hi)  u b(x,t)  is  the  potential  flow  part,  that  depends  only  from  the  boundary  conditions 
and  which  can  be  written  as 


V  x  u;,  (x,  t)  =  0 
V.ub  (x,f)  =  0 


=>  Ufe  (x,  t)  =  V(f  (x,  t)  with  A<j)  (x,  t )  =  0. 


Using  the  above  property  in  combination  with  Karhunen-Loeve  expansion  we  may  write 
every  sufficiently  smooth  (i.e.  spatially  differentiable  in  the  mean  square  sense)  random 
field  as 


u  (x,i;w)  =  Vxi)  (x,  t )  +  Vq  (x,  t )  +  V<f>  (x,  t)  (2-2) 

OO 

+  'YlYi  (*; w)  [V  x  (x’  *)  +  (x,  t)  +  V (pi  (x,  f)] 

i= 1 

Such  a  random  field  that  is  divergence-free  will  satisfy  the  equation 

V.u  (x,  t;  u)  =  0.,  x  e  D  C  M3,  UT,wGS1 
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Using  the  expansion  (2.2)  we  obtain 


V.u  (x,  t;  uj)  =  A q  (x,  t)  +  (f ;  a>)  Ag?:  (x,  t)  =  0  (2.3) 

i=l 

Since  the  above  equation  holds  for  arbitrary  wed  the  above  quantity  is  always  zero  if  and 
only  if  the  mean  and  the  stochastic  modes  are  divergence  free 

Aq  (x,  t)  =  Aqi  (x,i)  =  0,  i  =  1, .... 

In  addition  here,  the  last  set  of  equations  is  accompanied  by  homogeneous  boundary  con¬ 
ditions  since  the  non- homogeneous  components  are  satisfied  by  the  fields  (j),  fa,  i  =  1,.... 
Therefore  we  will  need  to  have  here: 

g(x,f)  =  g*(x,f)  =  0,  i  =  1,.... 


for  all  x  G  D  C  R3 ,  t  €  T. 

Using  similar  arguments  for  the  case  of  irrotational  or  potential  flow  fields,  i.e.  by  taking 
the  curl  or  the  divergence  of  the  Karhunen-Loeve  expansion,  we  obtain  the  following  result. 


Theorem  9  Let  u  (x,  t;  uj)  be  a  random  field.  Then  the  following  statements  are  true: 
i)  if  V.u  (x,  f;  uj)  =  0,  uj  £  U  then  the  flow  can  be  expanded  as 


u  (x,  t]  uj)  =  V  x  if  (x,  t)  +  V(jj  (x,  t) 


+  ]Yi  (f;  uj)  [V  x  fa  (x,  t)  +  Vfa  (x,  t)] 

i— 1 


(2.4) 


where  A (x,  t)  =  Afa  (x,  t)  =  0,  i  =  1, .... 

ii)  i/Vxu  (x,  t;w)  =  0,w£Sl  then  the  flow  can  be  expanded  as 

u  (x,  f;  cw)  =  Vg  (x,  f)  +  V<(>  (x,  f)  (2-5) 

OO 

+  (*;  w)  [Vg*  (x,  t)  +  Vfa  (x,  t.)} 

i—  1 

where  Afl  (x,  t)  =  Afa  (x,  t)  =  0,  i  =  1, .... 

in)  i/Vxu  (x,  t;  uj)  =  0,  and  V.u  (x,  t;  uj)  =  0  uj  €  fl  then  the  flow  can  be  expanded  as 

OO 

u  (x,  t]  uj)  =  Vfl  (x,  t)  +  {t;  uj)  Vfa  (x,  t) .  (2.6) 

1=1 

where  A<f>  (x,  t)  =  Afa  (x,  t)  =  0,  i  =  1, .... 


Moment  properties  of  random  flows  with  given  hydrodynamic  characteristics 

We  shall  now  use  the  representations  derived  in  the  last  Theorem  to  derive  properties  for  the 
covariance  operator  that  is  associated  with  the  random  field  u  (x,  t;  uj)  .  Using  representation 
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(2.4)  for  divergence- free  random  fields  we  will  have  the  covariance  function  given  by 


oo  oo 

=  iYk  (*;  w)  Yi  (t;  w)]  [V  x  i)k  (x,  t)  +  V4  (x,  t)]  [V  x  ijjl  (y,  t)  +  V(j)i  (y,  i)]T 

/c=l  Z=1 
oo 

=  [V  X  V’fc  (x,  i)  +  V0fc  (x,  t)]  [V  x  ipk  (y,  f)  +  (y,  t)]T 

k=  1 

We  also  have 

div  [V  x  ^ k  (x,  t)  +  (x,  f)]  =0,  k  =  1, ... 

Therefore,  we  obtain  the  following  property  of  covariance  functions  characterizing  incom¬ 
pressible  random  fields 


Vx.Cu(,t;w)u(,t;h;)  (x,y)  =  Vy.C^(.it;w)u(.)t;w)  (x,y)  =  0 
or  in  index  notation 

d  d 

'Q^~Y'Ui(-,t;u/)Uj(-,t;Lu)  (x>y)  =  (x,y)  =  0 

where  Vx  denotes  the  divergence  operator  with  respect  to  the  spatial  variables  x  (note  that 
x,y  are  3-dinrensional  vectors  representing  spatial  variables). 

Using  similar  arguments  we  have  the  following  property  for  irrotational  fields 

Vx  x  Cu(,t;w)u(,t;w)  (x,y)  =  Vy  x  (x,  y)  =  03x3 

or  in  index  notation 

d  d 

£i^k^x (x>  y)  =  £iji ~Q^~Euk{-)t-,u})Ui{-,t-,u)  (x,y) =  o 

where  is  the  Levi-Civita  symbol.  Using  these  results  we  can  generalize  the  above 
arguments  and  derive  similar  properties  for  higher  order  moments  of  random  fields. 

Monte  Carlo  simulation  of  random  flows  with  given  hydrodynamic  characteris¬ 
tics 

As  we  saw,  the  Karhunen  Loeve  expansion  separates  the  stochastic  from  the  spatial  struc¬ 
ture  of  the  problem  allowing  us  to  represent  uncertainty  very  effectively.  This  representa¬ 
tion  property  can  be  used  for  the  efficient  simulation  of  random  fields  with  given  properties 
such  as  incompressibility.  In  the  more  traditional  approach  for  sample  generation  of  ran¬ 
dom  fields,  the  spectral  decomposition  method  is  used  (see  e.g.  [115]).  In  this  framework 
the  statistics  of  the  random  field  are  assumed  to  be  Gaussian  with  spatially  homogeneous 
character.  This  assumption  allows  for  the  description  of  the  second-order  statistics  of  the 
random  field  through  the  spatial  Fourier  transform  of  the  covariance  operator.  The  gen¬ 
eration  of  samples  is  then  based  on  the  decomposition  of  the  full  spectrum  function  into 
smaller  (narrow  banded)  components  each  one  having  random  phase  which  is  uniformly 
distributed.  This  technique  is  very  popular  for  the  simulation  of  random  water  waves  (see 
e.g.  [106]). 
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Using  the  Karhunen  Loeve  expansion,  in  the  time-dependent  form  presented  above,  we 
may  generate  samples  by  performing  Monte-Carlo  simulation  to  the  random  coefficients 
Yi  (t;  uj)  and  with  fixed  modes  u?  (x,  t)  that  respect  a  priori  the  spatial  characteristics  of 
the  random  field  such  as  divergence-free  or  rotational-free  but  also  the  time-dependent 
characteristics  of  the  problem.  We  emphasize  that  through  this  approach,  no  assumptions 
are  required  for  the  spatial  characteristics  of  uncertainty,  such  as  homogeneity.  Moreover, 
the  non-Gaussian  character  of  the  random  field  is  fully  respected  since  this  is  expressed 
completely  through  the  random  simulation  of  the  scalar  coefficients  Yt  (t;  uj)  .  This  is  not 
the  case  for  other  approaches  used  such  as  the  POD  method  where  the  field  is  represented 
as 

S 

u  (x,  t;u)  =  u  (x)  +  ^ Yi  (t\  w)  u i  (x) 
i= 1 

or  the  polynomial-chaos  method  where  the  following  representation  is  used 

S 

u  (x,  t;uj)  =  u  (x,  t)  +  'Y2Xi  u*  (Xlt)  ' 
i= 1 

In  both  of  the  above  cases  the  lack  of  time  dependence  on  either  the  deterministic  fields  or 
the  stochastic  coefficients  restrict  the  representation  capabilities.  This  is  not  the  case  for 
the  general  Karhunen  Loeve  expansion  where  all  quantities  evolve  with  time,  allowing  for 
very  efficient  representation  of  any  given  field.  These  properties  will  also  be  of  particular 
importance  for  the  generation  of  samples  used  to  represent  the  initial  conditions  for  evolution 
problems  as  we  shall  see  in  the  next  chapter. 
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Chapter  3 

Evolution  of  stochastic  fields 


Abstract 

In  this  chapter  we  derive  an  exact,  closed  set  of  evolution  equations  for  general  continu¬ 
ous  stochastic  fields  described  by  a  Stochastic  Partial  Differential  Equation  (SPDE).  By 
hypothetizing  a  decomposition  of  the  solution  field  into  a  mean  and  stochastic  dynamical 
component,  we  derive  a  system  of  field  equations  consisting  of  a  Partial  Differential  Equa¬ 
tion  (PDE)  for  the  mean  field,  a  family  of  PDEs  for  the  orthonormal  basis  that  describe 
the  stochastic  subspace  where  the  stochasticity  ‘lives’  as  well  as  a  system  of  Stochastic  Dif¬ 
ferential  Equations  that  defines  how  the  stochasticity  evolves  in  the  time  varying  stochastic 
subspace.  These  new  evolution  equations  are  derived  directly  from  the  original  SPDE,  using 
nothing  more  than  a  dynamically  orthogonal  condition  on  the  representation  of  the  solution. 
If  additional  restrictions  are  assumed  on  the  form  of  the  representation,  we  recover  both  the 
Proper- Orthogonal-Decomposition  equations  and  the  generalized  Polynomial-Chaos  equa¬ 
tions.  The  material  presented  in  the  chapter  is  part  of  the  article  Sapsis  and  Lermusiaux, 
2009  [129]. 

3.1  Introduction 

In  the  past  decades  an  increasing  number  of  problems  in  continuum  theory  have  been 
treated  using  stochastic  dynamical  theories.  Applications  of  this  kind  include  problems 
where  even  though  the  details  of  the  response  maybe  different  over  various  runs,  their 
statistical  properties  obtained  by  averaging  over  appropriate  spatial  and  temporal  scales 
(‘spectral  windows’,  [105])  remain  similar.  To  this  end  one  may  decide  to  use  the  stochastic 
framework  to  model  the  effect  of  the  smaller  and/or  faster  scales  of  the  system  response  as 
well  as  the  effect  of  larger  and/or  slower  scales  of  the  boundary  conditions.  Such  problems 
are  mainly  described  by  stochastic  partial  differential  equations  (SPDEs)  and  they  arise 
in  a  number  of  areas  including  fluid  mechanics,  elasticity,  and  wave  theory  to  describe 
phenomena  such  as  turbulence  ([9],  [102],  [154],  [46],  [3],  [46]),  random  vibrations  ([137], 
[86],  [118]),  flow  through  porous  media  ([16],  [165]),  and  wave  propagation  through  random 
media  ([135],  [68],  [45]).  This  is  but  a  partial  listing  of  applications  and  it  is  clear  that  almost 
any  phenomenon  described  by  a  field  equation  has  an  important  subclass  of  problems  that 
may  profitably  be  treated  from  a  stochastic  point  of  view.  This  includes  problems  for  which 
the  dynamics  is  not  fully  resolved  or  not  sufficiently  known  to  warrant  solely  a  deterministic 
approach  as  well  as  problems  for  which  initial,  boundary  or  parametric  uncertainties  are 
significant. 
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A  basic  goal  of  uncertainty  quantification  is  to  estimate  joint  probability  distributions 
for  the  field  variables,  given  the  probabilistic  information  for  the  initial  state  and  forcing  of 
the  system  as  well  as  for  the  SPDE  random  coefficients.  A  complete  probabilistic  description 
of  the  response  would  either  require  the  knowledge  of  the  response  characteristic  functional 
or  equivalently  the  knowledge  of  the  whole  Kolmogorov  hierarchy  of  the  joint  probability 
distributions  of  the  response  stochastic  fields  at  any  collection  of  time  instances  and  spatial 
locations  ([16],  [67]).  Given  the  SPDE  that  governs  the  system,  it  was  first  shown  by  Hopf 
[60]  that  for  the  stochastic  Navier  Stokes  equations,  a  functional  differential  equation  can  be 
derived  that  governs  the  characteristic  functional  for  the  response.  His  approach  was  later 
adapted  to  the  problems  of  stochastic  wave  propagation  by  Tatarskii  [141]  and  Lee  [72], 
This  approach,  known  as  the  statistical  approach  to  turbulence,  has  also  been  developed 
further  by  many  authors  (see,  e.g.,  [85],  [44],  [69])  and  provides  with  infinite  dimensional 
transport  equations  for  the  characteristic  functional  that  characterizes  the  stochastic  solu¬ 
tion.  Even  though  these  functional  equations  contain  the  full  probabilistic  information  for 
the  dynamical  system  and  their  derivation  from  the  SPDE  is  straightforward  ([16]),  their 
infinite  dimensional  character  prevents  from  a  feasible  method  of  solution. 

The  Monte-Carlo  simulation  technique  is  a  more  practical  method  that  can  be  readily 
applied  to  solve  such  problems  to  an  arbitrary  degree  of  accuracy,  provided  a  sufficiently 
large  number  of  samples  is  used.  During  the  past  years  significant  advances  have  been 
made  in  improving  the  efficiency  of  Monte-Carlo  schemes.  This  includes  new  sequential 
Monte-Carlo  methods  (e.g.  Particle  Filters,  see  e.g.  [38])  where  the  probability  density 
function  of  the  response  is  approximated  by  a  mixture  of  weighted  Dirac  functions.  A  most 
recent  development  is  the  usage  of  a  mixture  of  weighted  Gaussian  kernels  instead  of  Dirac 
functions  to  provide  a  more  reliable  representation  of  the  response  pdf  ([30], [150]).  In  recent 
years,  such  particle  filters  and  their  variants  have  been  applied  to  stochastic  estimations 
in  various  fields  including  ocean  and  atmosphere  dynamics  ([61])  and  structural  dynamics 
([29]). 

Another  approach  is  based  on  the  generation  and  evolution  of  an  optimal  set  of  input 
samples  of  reduced  dimensionality  such  that  the  scales  and  dynamical  processes  where  the 
dominant,  most  energetic,  uncertainties  occur  are  continuously  spanned.  This  is  motivated 
by  the  multi-scale,  intermittent,  non-stationary  and  non-homogeneous  uncertainty  fields 
for  ocean  dynamics  (e.g.  [77]).  The  methodology,  referred  to  as  Error  Subspace  Statistical 
Estimation  (ESSE)  ([81], [74], [76], [79]),  uses  a  Karhunen  Loeve  (KL)  expansion  but  with 
time- varying  and  adaptive  basis  functions  (see  Chapter  2,  Theorem  7).  The  functions  are 
evolved  using  stochastic,  data- assimilative,  ensemble  predictions  initialized  by  a  multi-scale 
scheme  and  evolved  through  a  Monte-Carlo  approach.  Similar  ideas  have  been  later  applied 
to  the  modeling  of  diffusion  processes  in  random  heterogenous  media  ([47],  [48]).  These 
methods  approximate  the  response  pdf  without  making  any  explicit  assumption  about  its 
form  and  can  thus  be  used  in  general  nonlinear,  non-Gaussian  systems.  However,  a  major 
issue  is  that  the  evolution  of  the  dominant  uncertainties  is  through  a  Monte-Carlo  scheme 
and  therefore  a  larger  number  of  samples  can  still  be  required  for  accurate  prediction. 

Order  reduction  methods  have  also  been  utilized  to  derive  reduced-order  models  which 
have  lower  complexity  relative  to  the  original  SPDE  model  and  which  reveal  the  under¬ 
lying  structure  of  the  system  dynamics.  A  classical  approach  is  the  statistical  technique 
of  Karhunen-Loeve  expansion  or  Proper  Orthogonal  Decomposition  (POD)  (see  [109],  [89], 
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[59])  where  the  response  of  the  dynamical  system  is  usually  assumed  to  have  the  form 

S 

u  (x,  t\  u>)  =  ^^A,;  (i;  lu)  u.j  (x) ,  10  €  fl  (3.1) 

i— 1 

where  Aj  ( t ;  oj)  are  stochastic  processes  and  the  family  Ui  (x)  are  time-independent  func¬ 
tions  computed  from  data  collected  in  the  course  of  experiments  or  from  direct  numerical 
simulations.  Specifically,  Ui  (x)  s  are  orthonormal  fields  that  provide  an  optimal  modal 
decomposition  in  the  sense  that  a  finite  collection  of  these  modes  can  capture  the  domi¬ 
nant  components  of  the  complete  infinite-dimensional  process.  A  Galerkin  projection  of  the 
original  governing  equations  to  the  low-dinrensional  subspace  identified  by  the  POD  basis 
functions  Ui  (x)  provides  the  reduced  order  evolution  equations  for  the  unknown  stochastic 
coefficients  A;  (t;  u>).  Note  that  POD  is  most  commonly  used  in  the  deterministic  frame¬ 
work  to  derive  reduced  order  dynamical  equations.  In  this  case  the  averaging  is  usually 
performed  over  time  and  more  rarely  over  realizations.  The  POD  concept  has  been  applied 
to  a  wide  range  of  areas  such  as  turbulence  ([89],  [134],  [19],  [59]),  and  control  of  chemical 
processes  ([50],  [132],  [166]).  However,  the  main  drawback  of  the  POD  method  is  that  the 
basis  functions  are  chosen  a  priori  and  therefore  may  not  be  able  to  efficiently  represent  the 
evolving  responses  generated  by  nonlinear  dynamical  processes. 

Another  main  approach  is  the  Polynomial  Chaos  (PC)  expansion  pioneered  by  Ghanern 
and  Spanos  [52]  in  the  context  of  solid  mechanics.  It  is  based  on  the  original  theory  of  Wiener 
on  polynomial  chaos  ([156],  [25],  [157]).  The  stochastic  field  describing  the  system  response 
is  treated  as  an  element  in  the  Hilbert  space  of  random  functions  and  is  approximated  by  its 
projection  onto  a  finite  subspace  spanned  by  orthogonal  polynomials.  Specifically,  instead 
of  imposing  a  representation  for  fixed  fields  u*  (x)  as  for  the  POD  method,  the  stochastic 
processes  Aj  (i;  a;)  are  spectrally  represented  in  terms  of  fixed  multi-dimensional  Hermite 
polynomials, 

S 

u  (x,  t;  u)  =  y (C  M)  u»  (x,  t ) ,  w  G  O  (3.2) 

i= 1 

where  are  orthogonal  polynomials  and  C  (w)  are  given  random  variables.  A  Galerkin 
projection  of  the  governing  equations  to  the  low-dinrensional  subspace  defined  by  the  <hjS 
transforms  the  original  SPDE  to  a  set  of  coupled  deterministic  PDEs  for  the  unknown 
family  Uj  (x,  t).  The  method  has  been  applied  to  a  series  of  applications  including  fluid 
mechanics  ([31],  [91],  [162],  [66],  [34]),  structural  mechanics  ([52],  [51],  [130]),  and  wave 
propagation  in  random  media  ([92]).  Although  for  any  arbitrary  random  process  with 
finite  second-order  moments,  the  PC  expansion  converges  in  accord  with  Canreron-Martin 
theorem  [25],  it  has  been  demonstrated  that  the  convergence  rate  is  optimal  (i.e.  a  given 
error  tolerance  is  achieved  with  the  smallest  number  of  terms  in  the  series)  for  Gaussian 
processes  [88],  while  for  other  types  of  processes  the  convergence  rate  may  be  substantially 
slower.  A  recent  development  by  Xiu  and  Karniadakis  ([161],  [162])  proposes  a  generalized 
PC  expansion  where  basis  functions  from  the  Askey  family  of  hypergeometric  polynomials 
are  used.  It  is  shown  that  suitable  basis  functions  different  from  the  Hermite  polynomials 
can  increase  substantially  the  rate  of  convergence.  Depending  on  the  form  of  the  SPDE,  its 
stochastic  coefficients,  and  the  initial  stochastic  conditions,  there  is  an  optimum  choice  of 
basis  functions  through  which  an  optimum  rate  of  convergence  can  be  achieved.  However, 
this  choice  must  be  made  a  priori  and  this  can  be  a  very  challenging  task  especially  for 
non-stationary  complex  dynamical  systems  with  large  number  of  degrees  of  freedom  (e.g. 
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atmospheric  or  oceanic  dynamics). 

Here  we  will  utilize  the  general  Karhunen-Loeve  expansion  presented  in  Chapter  2  (The¬ 
orem  7  and  references  therein), 

S 

u  (x,  i;  co)  =  u  (x,  t)  +  ^^Yj  (f ;  lo)  u,  (x,  t) ,  wed  (3.3) 

i= 1 

in  order  to  derive  evolution  equations  for  the  Y)  (f;  cw),  u  (x,  t )  and  u,  (x,  f)  without  making 
any  assumptions  on  their  form:  the  original  SPDE  governing  u  (x,  t ;  lo)  is  the  only  infor¬ 
mation  utilized.  Using  a  new  dynamical  orthogonality  condition  for  the  fields  Uj  (x,  t),  we 
overcome  the  redundancy  of  representation  (3.3)  and  derive  an  exact  set  of  evolution  equa¬ 
tions  that  has  the  form  of  an  s— dimensional  stochastic  differential  equation  for  the  random 
coefficients  Y)  (f;  co)  coupled  with  s+1  deterministic  PDEs  for  the  fields  u  (x,  t )  and  u.;  (x,  t ) , 
where  s  is  the  number  of  modes  that  we  retain  in  representation  (3.3).  In  this  way,  the  basis 
that  describes  the  stochastic  subspace  is  dynamically  evolved  and  is  not  chosen  a  priori:  it 
adapts  to  the  stochasticity  introduced  by  the  stochastic  initial  conditions  and  coefficients, 
and  evolves  according  to  the  SPDE  governing  u  (x,  f;  co).  The  stochastic  coefficients  Y)  (t;  lo) 
are  also  evolved  according  to  dynamical  equations  derived  directly  from  the  original  SPDE 
allowing  us  to  use  any  SDE  numerical  scheme  for  their  solution  (e.g.  particle  methods). 
For  the  special  case  of  stochastic  excitation  that  is  delta  correlated  in  time,  i.e.  white  noise, 
an  equivalent  non-linear  Fokker-Planck-Kolmogorov  equation  describes  the  evolution  of  the 
joint  probability  density  function  for  the  stochastic  processes  Y)  (f ;  lo)  . 

The  derived  field  equations  are  consistent  with  the  dynamical  orthogonality  condition 
which  also  implies  the  preservation  of  the  classical  orthonormality  condition  for  the  fields 
Uj  (x,  t )  .  If  additional  suitable  assumptions,  either  on  the  form  of  the  fields  u  (x,  t )  and 
u*  (x,  t ),  or  on  the  form  of  Yj  (f;  lo)  are  utilized,  our  novel  equations  reproduce  the  reduced- 
order  equations  obtained  by  application  of  the  POD  or  PC  method,  respectively. 


3.2  Definitions  and  problem  statement 


Let  (f },£q,V )  be  the  probability  space  and  let  x  eD  C  Mn  denote  the  spatial  variable  and 
f  £  T  the  time.  In  applications,  the  most  important  cases  are  where  n  =  2,  3  therefore  in 
what  follows  we  will  assume  that  xGD  C  Mn,  n  =  2,  3.  The  set  of  all  continuous,  square 


integrable  random  fields,  i.e.  /  E u 


u  (x,  t;  lo)  u  (x,  t;  lo)  dx  <  oo  for  all  t  G  T  (where 


D 


denotes  the  complex  conjugate  operation)  and  the  covariance  operator 

Cu(-,t;w)v(.,s;w)  (X,  y)  =  (u  (x,  t]  co)  -  u  (x,  t) )T  (v  (y ,  s;  lo)  -  v  (y ,  s) )  ,  x,y  GD,  t,  s  £  T 

(3.4) 

form  a  Hilbert  space  ([120],  [135])  that  will  be  denoted  by  H. 

For  every  two  elements  u  (x,  t;co),  v  (x,  t;  co)  €  H  we  define  the  spatial  inner  product  as 


(u  (•,t;oo)  ,v  (•,t;u>)) 


j u  (x,  t;  (o)T 
D 


v  (x,  i;  lo)  dx 


where  the  integral  on  the  right  hand  side  is  defined  in  the  mean  square  sense  ([87]).  For 
the  case  where  the  integrands  are  deterministic  the  mean  square  integral  is  reduced  to  the 
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classical  Riemann  integral.  In  what  follows  we  will  use  Einstein’s  convection  for  summation, 
i.e.  ^^ciibi  =  dibi  except  if  the  limits  of  summation  need  to  be  shown.  A  double  index  that 

i 

is  not  summed-up  will  be  denoted  as  ajbi.  We  define  the  projection  operator  II  of  a  field 
u  (x,  f),x  gD  to  an  m— dimensional  linear  subspace  spanned  by  the  orthonormal  family 
{wj  (x,  t;  w)}”l1 ,  x  £D  as  follows 

m 

n{w,(x.t;w)}™  x  [u  (x,  t;  w)]=5^Wj  (x,  t;  u)  (w j  (•,  t;  u) ,  U  (•,  t;  w)) 

3= 1 

=W j  (x,  f;  w)  (Wj  (•,  t;  w) ,  u  (#,  t;  lo)) . 

The  SPDE  describing  the  system  evolution  is  assumed  to  have  the  form 

- — -  j. — -  =  C  [u  (x,  t;  to) ;  lo] ,  x6D,  i  £  T,  to  £  fl  (3.5) 

where  £  is  a  general  (nonlinear),  differential  operator.  Additionally,  the  initial  state  of  the 
system  at  to  is  described  by  the  random  field 


u  (x,  to!  w)  =  Uo  (x;  u) ,  x  £D,  lo  €  II  (3.6) 

and  the  boundary  conditions  are  given  by 

B  [u  (£,t;  cu)]  =  h  (£,f;  oj)  ,  lo  €  Q  (3.7) 

where  B  is  a  linear  differential  operator.  For  all  of  the  above  quantities  we  assume  that 
random  coefficients  have  statistical  moments  of  every  order. 


3.3  The  stochastic  subspace  and  the  dynamical  orthogonality 
condition 

As  we  saw  in  the  last  section  of  Chapter  2  Karhunen-Loeve  representation  provide  us  with 
a  tool  to  represent  every  random  field  u  (x,  t\  lo)  £  H  at  a  given  time  t  by  a  series  of  the 
form 

OO 

u  (x,  i;  to)  =  u  (x,  t)  +  (f ;  lo)  u,;  (x,  t) ,  to  £  fl  (3.8) 

i= 1 

where  u*  (x,  t )  are  the  eigenfunctions,  and  Yt  (f;  lo)  are  zero-mean,  stochastic  processes  with 
variance  Ew  [Y-2  ( t ;  cu)]  equal  to  the  corresponding  eigenvalue  A f  (t)  of  the  eigenvalue  prob¬ 
lem 

j C u(.,t)u(.,t)  (x,  y )  u*  (x,  t)  dx  =A?  (t)  u*  (y ,  i)  ,  y  £E>.  (3.9) 

D 

Moreover,  we  discussed  that  the  important  mass  of  the  probability  measure  can  always  be 
captured  by  a  finite  dimensional  subspace  in  the  sense  that  for  every  acr  >  0  there  is  always 
a  positive  integer  s  such  that  the  finite  truncation  of  (3.8)  that  contains  the  first  s  terms  is 
<rcr— close  (in  the  mean  square  sense,  i.e.  in  the  norm  ||-||2  =  (•,•})  to  the  field  u(x, t;co). 
Therefore,  every  random  field  u  (x,  f ;  lo)  £  H  can  be  approximated  arbitrarily  well,  by  a 
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finite  series  of  the  form 


S 

u  (x,  t]  u)  =  u  (x,  t )  +  ^ Yj  (t;  u)  (x,  t ) ,  w  G  H  (3.10) 

1=1 

where  s  is  a  sufficiently  large,  non-negative  integer.  Based  on  the  above  discussion  we 
define  the  stochastic  subspace  V5  =  span  {u,  (x,  i)}*=1  as  the  linear  space  spanned  by  the 
s  eigenfields  that  correspond  to  the  s  largest  eigenvalues.  A  graphical  interpretation  of  the 
stochastic  subspace  is  given  in  Figure  3-la  where  we  represent  the  space  Vs  as  a  plane  along 
which  the  probability  measure  presents  important  spread.  On  the  other  hand  along  any 
direction  included  in  Vg,  represented  as  the  red  axis,  the  spread  of  probability  is  smaller 
(in  the  mean  square  sense)  than  acr.  Hence,  V5  defines  the  appropriate  subspace  where 
the  stochasticity  of  the  random  field  ‘lives’  at  time  t,  following  ESSE  ideas  ([81,  75]).  Our 
goal  now  is  two-fold: 


1.  For  fixed  dimensionality  s  study  how  the  stochasticity  evolves  inside  V5.  More 
specifically  we  seek  the  equations  governing  the  evolution  of  the  stochastic  vector 
{Yj(t-MYj= v 

2.  Study  how  V5  evolves  inside  H  through  the  variation  of  the  basis  {u^  (x,  i)}^=1  . 

Clearly,  representation  (3.10)  with  all  quantities  (u  (x,i) ,  {uj  (x,  t) }*=1 ,  {Yj  (t;  w)}j=1) 
varying  is  redundant  and  therefore  we  cannot  derive  independent  equations  from  the  SPDE 
describing  their  evolution.  Hence,  it  is  essential  to  impose  additional  constraints  in  order 
to  get  a  well  posed  problem  for  the  unknown  quantities. 

To  this  end  we  examine  more  carefully  the  source  of  redundancy  in  representation  (3.10). 
Specifically,  we  notice  that  the  variation  of  the  stochastic  coefficients  {Yj  (t;u;)}^=1  can  ex¬ 
press  exclusively  the  evolution  of  uncertainty  within  the  stochastic  space  V5.  On  the  other 
hand,  by  varying  the  basis  {in,-  (x,  t)}^=1  we  can  express  both  the  evolution  of  uncertainty 
within  Vs  and  also  normal  to  Vs-  Therefore,  we  see  that  the  source  of  redundancy  comes 
from  the  evolution  of  uncertainty  that  can  be  described  by  both  the  variation  of  the  stochas¬ 
tic  coefficients  and  the  basis.  To  overcome  this  difficulty  we  need  to  restrict  the  evolution 
of  the  basis  {uj  (x,  t)}*=1  to  be  normal  to  the  space  Vs  since  the  evolution  within  Vs  can 
be  described  completely  by  a  rotation  of  the  stochastic  coefficients  as  it  is  shown  in  Figure 
3-lb.  The  above  requirement  can  be  elegantly  expressed  through  the  following  condition 

IVstt  |  dU,Qt ’  ^  1  ui  (*’  =  0  =  j  =  l,...,s.  (3.11) 


We  will  refer  to  the  above  condition  as  the  dynamically  orthogonal  (DO)  condition.  Note, 
that  the  DO  condition  implies  the  preservation  of  orthonormality  for  the  basis  {uj  (x,  f)}^=1 
since 


d 

dt 


(Uj  (»,t)  ,Uj  (#,t)) 


/  dui  (*,f) 

\  dt 


,u j  (*,f) 


+ 


duj  (*,t) 

dt 


To  summarize  the  above  discussion,  in  what  follows  we  will  use  the  DO  representation 
defined  by  equation  (3.10)  and  the  additional  properties 


1.  {Yj  (t;cu)}*=1  are  zero-mean  stochastic  processes. 
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2.  {uj(x,i)}^=1  are  deterministic  fields  satisfying  the  DO  condition  (3.11)  which  are 
initially  orthonormal,  i.e.  (u,;  (#,  to)  ,  u j  (#,  to))  =  <%. 


3.4  Dynamically  orthogonal  field  equations 

In  this  section  we  use  representation  (3.10)  to  derive  reduced  order  field  equations  describing 
the  mean  state  of  the  system,  its  stochastic  characteristics  and  their  interactions.  As  it 
is  proven  in  the  following  theorem,  the  DO  expansion  results  in  a  set  of  independent, 
explicit  equations  for  all  the  unknown  quantities.  In  particular,  using  the  DO  expansion 
we  reformulate  the  original  SPDE  to  an  s— dimensional  stochastic  differential  equation  for 
the  random  coefficients  Yt  (i;  u)  coupled  with  s  +  1  deterministic  PDEs  for  the  fields  u  (x,  t) 
and  Ui  (x,  t)  . 


Theorem  10  (DO  evolution  equations)  Under  the  assumptions  of  the  DO  representa¬ 
tion  the  original  SPDE  (3. 5)- (3. 7 )  is  reduced  to  the  following  system  of  equations 

tu  (•>  *; -  E“  [£  iu  (•>  w) ;  <dl , w  (•,  t)) ,  (3.12) 

— ^li  =  £“[£[u(x,t;a,);w]],  (3.13) 

=  nvi  [£“  [C  [u  (x,  t;  w) ;  u]  Y,  (t;  u>)]]  Cy‘,)yj(t)  (3.14) 

where  nv±  [F  (x)]  =  F  (x)-nVs  [F(x)]  =  F  (x)-(F(#)  ,uk  (•,t))ufc  (x,t)  and  CYi(t)Yj{t)  = 
E w  [Yi  (t\u)  Yj  (i;u;)].  The  associated  boundary  conditions  have  the  form 

u  (£,  t]  w)]  |€eSD  =  E“  [h  (£,  t;  w)] 

B  [u*  {£,t)]  =  E  [Yj  (t;  u)  h  (£,  i;  a;)]  ^Yi(t)Yj(t) 

and  the  initial  conditions  are  given  by 

Yi  (t0;  oj)  =  (u0  (•;  u)  -  u0  (•) ,  Uj0  (•)) 
u  (x,  t0)  =  u0  (•)  =  Eu  [u0  (x;  w)] 

Uj  (X,  t0)  =  Uj0  (x) 

for  all  i  =  1, ...,  s,  where  u;o  (x)  are  the  eigenfields  of  the  correlation  operator  Cu(.jto)u(.tto\ 
defined  by  the  eigenvalue  problem  (3.9). 


Proof:  First  we  insert  the  DO  representation  to  the  evolution  equation  (3.5).  We  obtain 
(using  Einstein’s  summation  notation  defined  in  Chapter  2) 

<9u(x,t)  dYj  (t;  u)  ,^.vU  >  *  (x4)  r[  ,  .  ,  , 

H - 77 - Uj  (x,  t)  +  Yi  (t;  us) - — - =  C  [u  (x,  t;  u) ;  uj\  .  (3.15) 


dt 


dt. 


By  applying  the  mean  value  operator  we  obtain  the  second  equation  of  the  theorem  (equa¬ 
tion  3.13),  i.e.  an  evolution  equation  for  the  mean  part  of  the  representation. 
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Figure  3-1:  a)  The  stochastic  subspace  V5  spanned  by  fields  u,  (x,  t)  that  corresponds  to 
important  variance,  b)  The  variation  of  the  Uj  (x,  t)  inside  Vs  can  always  be  covered  by  a 
rotation  of  the  stochastic  coefficients  Y{  (t\ uj)  . 
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By  considering  the  inner  product  of  the  evolution  equation  (3.15)  with  each  of  the  fields 
{uj  (x,  i)Y=1  we  have 


dUg*’^,ui  (•,*)^  +  (u*  (•>*)  >ui  (•>*))  +  ^  ^  >  uj  (•,*) 

=  (£  [u  (x,  t;  cu) ;  cu]  ,  Uj  (*,t))  . 


Now,  the  second  term  on  the  left  hand  side  vanishes  because  of  orthonormality  except  one 
term  for  which  i  =  j.  Moreover,  the  DO  condition  implies  that  the  third  term  vanishes 
completely.  Therefore  we  have  the  family  of  s  stochastic  differential  equations 


dYj  (t;  u)  /du  (•,  t)  ,  . 

dt  +  \  dt  ’ 


{£  [u  (#,  t;  u) ;  w]  ,  Uj  (•,  t)) ,  j  =  1, ...,  s. 


Note  that  by  using  (3.13)  or  by  applying  the  mean  value  operator  to  the  above  equation, 
we  obtain 


/  <9u  (m,t) 
\  dt 


Eu[(£  [u  (•,i;u;);u;] 


%•  (•,*))],  i  =  i,...,s. 


The  quantity  ^  ,  u  ;  (•,  t)^  expresses  the  variation  of  u  towards  directions  of  the  stochas¬ 

tic  subspace  Vg.  Hence,  the  equation  for  Y  (i;a>)  will  take  the  final  form  (3.12). 


As  a  next  step,  we  multiply  equation  (3.15)  with  Y)(i;u>)  and  apply  the  mean  value 
operator  to  get 


Eu 


dYj  (t;  to) 
dt 


Yj  (t;  u) 


■  (~x  t'] 

ut  (x,  f)+F"  [Yi  (t;  w)  Y}  (t;  cu)]  ^  j  =  E“  [C  [u  (x,  t;  w) ;  w]  Yj  (*;  <*>)] 


which  can  be  written  as 
duj  (x,  t) 


'Yi(t)Yj(ty 


dt 


+  Cd^t)Yj{t)u i  (x,  t)  =  Eu  [£  [u  (x,  t\ u) ;  uj\  Yj  (t;  w)]  (3.16) 


dt  1 0 


where  C Yi(t)Yj(t)  =  -EY  [Y)  (i;  cu)  Yj  (i;  w)].  By  considering  the  inner  product  of  the  last 
equation  with  the  field  Ufc  (x,  t )  ,  and  using  DO  condition,  we  obtain  an  exact  expression  for 

Cd-WYA« 

Cdvfc(t)  vm  =  Eu  [{£  [u  (x,  t\  u) ;  u\ ,  ufc  (#,*))  Y)  (t;cu)]  .  (3.17) 

dt  *0  \l) 

Note  that  this  result  (3.17)  can  also  be  obtained  from  the  definition  of  Cdrfc(t)v  ,^and 

dt  D  W 

from  equation  3.12.  Now,  inserting  the  last  expression  to  equation  (3.16)  will  result  in  the 
equation 

f)yi  ■  ( v 

CYid)Yj(t) — lQt  '  =  E“  tu  (x’  (*; W)1  -  nvs  [£“  [£  [U  (x,  i;  uj) ;  w]  Yj  (i;  w)]] 


=  nv^  [£  [u  (x,  w) ;  w]  Yj  (f;  w)]] 


where 


nvx  [F  (x)]  =  F  (x)  -  (F  (•) ,  Ufc  (#,  t))  Ufc  (x,  t) 
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Moreover,  since  C Yjt)Y(t)  is  positive-definite  it  can  always  be  inverted  and  therefore  we 
obtain  the  final  expression  (3.14)  for  the  evolution  of  the  fields  u*  (x,  t). 

Finally,  by  applying  the  mean  value  operator  on  equation  (3.7)  for  x  6  dD  we  obtain 
the  boundary  condition  for  the  evolution  of  the  mean  field 

B  [u  (£,  t ;  cu)]  \^aD  =  E ,w  [h  (£,  t;  a;)]  . 

Additionally,  by  multiplying  equation  (3.7)  with  Yj(t\ui)  and  applying  the  mean  value 
operator  we  obtain  for  x  £  dD 

Ew  =  Cy.(t)yi(t)  B[ui($,t)]|Ce8£). 

Therefore, 

B  [Ui  (£,  t)}\iedD  =  E“  [Yj  (f,  u)  h  (£,  t-  a;)]  C"^. 

The  initial  conditions  for  the  quantities  involved  are  found  by  approximating  the  initial 
field  uo  (x;  uj)  by  a  truncated  Karhunen-Loeve  expansion  containing  s  terms.  Therefore, 
the  initial  conditions  Ujo  (x)  for  the  fields  u,  (x,  t)  will  be  the  s  most  energetic  eigenfields 
of  the  correlation  operator  Cu(.^0)u(.^0)  defined  by  the  eigenvalue  problem 

j cu(;t0)u(;t0)  (x,  y)  Uj0  (x)  dx  =A?Uj0  (y) ,  y  &D. 

D 

The  initial  conditions  for  the  stochastic  coefficients  Yt  (£;  uj)  will  be  given  by  the  projection 
of  the  field  uo  (x;  to)  —  uo  (x)  to  the  orthonormal  eigenfields  Ujo  (x)  as  follows 

Yi  (t0;  uj)  =  (u0  (•;  u)  -  u0  (•)  ,  Uj0  (•)) , 

and  the  initial  condition  for  the  mean  field  will  be  given  by  u  (x,  to)  =  uo  (x)  =  [uo  (x;  w)]  .  □ 

As  it  can  be  easily  verified  the  evolution  equations  derived  above  are  consistent  with 
the  DO  condition  that  was  initially  assumed.  Also,  the  initialization  procedure  for  our  DO 
field  equations  follows  the  multivariate  ESSE  approach,  e.g.  [75,  78]. 

It  should  be  emphasized  that  the  knowledge  of  the  full  set  of  quantities  associated  with 
the  DO  expansion,  i.e.  {Yj  (f;  o;)}^=1  ,  u  (x,  t)  ,  and  Uj  (x,  t)  can  lead,  through  simple  random 
variable  transformations  ([109]),  to  analytic  expressions  of  any  statistical  quantity  of  interest 
(e.g.  pdfs  of  velocity  field  at  particular  positions  of  the  domain,  spectral  representations  of 
the  stochasticity  etc.)  in  terms  of  these  DO  expansion  quantities  (see  Chapter  5). 


3.4.1  The  case  of  independent  increment  excitation  (white  noise) 

A  special  class  of  SPDE  of  great  importance  is  the  case  where  the  operator  C  can  be  linearly 
split  to  a  deterministic  part  and  to  a  stochastic  part  having  the  form  of  derivative  of  an 
independent  increment  process  ([116]),  e.g.  Brownian  motion  or  Poisson  process.  More 
specifically  we  consider  the  special  case  of  a  system  excited  by  an  independent  increment 
(with  respect  to  time)  stochastic  process  and  having  deterministic  boundary  conditions, 
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described  by  the  evolution  equation 


<9u  (x,  t\  lu) 

Ft 

u  (x,t0;w) 
B[u  (£,t;u)] 


V  [u  (x,  t\  u;)]  +  Y*r  (x,  t ) - r-^- — -,  x  6D,  t€.T, 

r= 1 

uo  (x;  uj)  ,  x£D,  w6!1 

ho  (£,  t) ,  £  G  31?,  wed 


(3.18) 


where  V  is  a  deterministic,  differential  operator,  h/5  (£,  t)  is  a  deterministic  quantity  defining 
the  boundary  conditions,  {<&r  (x,  i)},f=1  are  deterministic,  sufficiently  smooth  fields,  and 
{Wr  (t;  cu)}r=1  are  taken  for  simplicity  to  be  independent  Brownian  motions  (although  the 
proof  follows  exactly  the  same  steps  for  general  independent  increment  processes).  In  this 
case  an  alternative  description  of  the  stochasticity  inside  V5,  can  be  given  in  terms  of  the 
probability  density  function  /y  (2/1 , 2/2, ... ,ys,t )  for  the  stochastic  vector  {Yj  (t;cu)}*=1  .  For 
simplicity  in  what  follows  we  will  also  use  the  following  notation 


V  [U  (x,  t)  ,  Y  (t;  w)]  ee  V 


s 

U  (x,  t )  +  ^Yj  (t-  (Jj)  Uj  (x,  t) 
i= 1 


=  V[u  (x,t;w)] 


with  U  (x,  t)  referring  to  the  s  + 1  fields  u  (x,  t)  ,  {uj  (x,  t)  }*=1  •  We  then  obtain  the  following 
result. 


Corollary  11  Under  the  assumption  of  the  DO  representation  the  SPDE  (3.18)  is  reduced 
to  the  following  system  of  equations 


dfy 

dt 


+  X 


_d_ 

% 

1  d 2 


/Y  (d[U  (rn,t) ,  y]  -  /  /y  (v,t)V[U  (#,t)  ,u]du,  u*  (#,t) 

'  Rs  ' 

[fYQij  (*)] 


(3.19) 


<9u  (x,  t) 

Ft 


du i  (x,  t) 
dt 


2  %% 

=  f  fv  (v,  t )  £>  [W  (x,  t) ,  u]  du, 


n 


V 


VjfY  (u,  t)  V  \U  (x,  t ) ,  u]  du 


(3.20) 


(3.21) 


where  Qtj  (t)  =  (&r  (•,  t ) ,  u*  (•,  i))  ($r  (#,  f)  ,  uj  (#,  f))  and  C Yi(t)Yj{t)  =  /  vivjfY  (v,  t)  dv. 

Rs 

The  associated  boundary  conditions  have  the  form 

B[u(f,i;w)]| $e9D  =  h£>  (£>*) 

S  [Ui  (£,  t)]|ge0£)  =  0 
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and  the  initial  conditions  are  given  by 


/y  (y,  to)  =  /y„  (y) 
u  (x,  to)  =  Eu  [u0  (x;  w)] 
u*  (x,  t0)  =  u,:0  (x) 

for  all  i  =  l,...,s,  where  u,;o  (x)  are  defined  in  Theorem  1  and  /y0  (y)  is  the  probability 
density  function  associated  with  the  random  vector  Yi  (to;  u)  =  (uo  (•;  uf)  —  uo  (•)  ,  u,;o  (•)) . 

Proof:  By  using  the  special  form  of  the  SPDE  (3.18)  and  the  zero  mean  property  of 
the  Brownian  motion  we  obtain  from  equation  (3.12) 

dYi%U)  =  \U  (*,  *)  - Y  ft ")]  -  [V  [l U  (x,  f)  ,  Y  (t;  u)}] ,  Ui  (.,  t)) 

,  T  .  .  .  . .  dWr  (t;  lo) 

+  ($r  («,t)  ,Uj  (•,*)) - — - . 

This  last  equation  is  an  Ito  stochastic  differential  equation  and  can  be  written  equivalently 
as  a  transport  equation  for  the  probability  density  function  /y  (yi,  V2,  ■■■,  Vs,  t) 

^  [/y  «P  [W  (.,  t) ,  y]  -  £"  [P  [W  (.,  t) ,  y]] ,  u,  (.,  t))}}  (3.22) 

i  d 2 

=  ^  ^  (*>  ^  ’  U*  (#>  *))  (#.  ^)  >  UJ  (»,  *))]  ' 

Note,  that 

[P  [U  (x,  t) ,  Y  (f;  w)]]  =  f  /Y  (f ,  i)  P  [W  (x,  f) ,  u]  du 

Rs 

and  hence  equation  (3.19)  follows.  Equation  for  the  mean  field  follows  directly  from  the 
corresponding  equation  (3.13)  and  the  zero-mean  property  of  the  Brownian  motion.  Finally, 
using  equation  (3.14)  we  obtain 

=  nVi  [Eu  [ V  [u  (x,  t;  u,)]  Y}  (t;  u,)]]  C£(Wj(t) 

+  nVi  (.,())£"  c$mm. 

The  second  term  on  the  right  hand  side  vanishes  due  to  the  non-anticipative  property  of 
Brownian  motion  ([135]).  Moreover, 

Eu  [P  [u  (x,  t;  u;)]  Yj  (f;  cu)]  =  /  VjfY  (v,  t )  P  [Zd  (x,  f) ,  v]  dv. 

Rs 

Additionally,  for  the  boundary  conditions,  we  obtain  from  Theorem  1 

B  [Ui  (£,  m^8D  =  Eu  [Yj  (t;  u)  hD  (£,  t)}  c^(t)y.(t)  =  hD  (£,  t)  EP  [Yj  (t;  W)]  =  0. 

Therefore,  equation  (3.21)  follows.  Finally  the  initial  conditions  are  defined  as  in  Theorem 
1  with  the  stochastic  vector  Yi  (to]Lo)  =  (uq  (•;  co)  —  uq  (•)  ,u*o  (•))  described  now  by  the 
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associated  probability  density  function  /y0  (y)  .  □ 

Note  that  a  transport  equation  for  the  probability  density  function  can  also  be  obtained 
for  the  case  of  general  time  correlation  structure  for  the  excitation  using  recent  results  for 
stochastic  dynamical  systems  ([65],  [127]). 

3.5  Consistency  with  existing  methodologies 

The  derivation  of  our  new  dynamically  orthogonal  field  equations  (3.12)-(3.14)  was  based 
exclusively  on  the  representation  of  the  solution  by  the  DO  expansion.  In  what  follows,  we 
show  that  by  imposing  additional  restrictions  on  the  representation,  either  those  of  the  PC 
or  the  POD  expansion,  we  recover  the  set  of  equations  that  are  obtained  for  each  of  these 
expansions.  Therefore,  the  DO  field  equations  (3.12)-(3.14)  can  be  considered  as  a  general 
methodology  that  unifies  two  of  the  most  important  and  widely  used  methods  for  evolving 
uncertainty  in  stochastic  continuous  systems  governed  by  a  SPDE  or  a  system  of  SPDEs. 

3.5.1  Generalized  polynomial  chaos  expansion 

In  the  generalized  PC  method,  introduced  by  Xiu  and  Karniadakis  [161]  the  stochastic 
processes  {Yj  (i;u;)}*=1  are  chosen  a  priori  and  often  fixed  in  time,  based  on  the  statistical 
characteristics  of  the  system  input  process.  Specifically,  the  stochastic  processes  are  chosen 
to  have  the  statistically  time-independent  form 

Yj(t-,u)  =  $j(C(w))  (3.23) 

where  are  orthogonal  polynomials  from  the  Askey-scheme  and  ( (cu)  are  given  random 
variables  [161].  In  this  case,  the  following  orthogonality  relation  in  the  random  space  holds 
between  the  stochastic  coefficients 

E“  [*i  (C  M)  ^  (C  M)]  =  ET  [$i  (C  M)2]  <%. 

The  reduced  order  PC  equations  (e.g.  [91], [162], [21], [66], [103])  are  usually  derived  by  substi¬ 
tuting  a  representation  as  (3.10)  but  with  the  stochastic  coefficients  given  by  (3.23)  to  the 
SPDE  (3.5)  and  then  projecting  it  to  the  stochastic  orthogonal  basis  functions  4>j  (£  (ui)) 
using  the  inner-product.  Following  these  steps  we  obtain 

E“  [*i  (C  M)2]  du^2t]Uj)  =  E“  [C  [u  (X,  t;  u) ;  a;]  (C  (u))}  .  (3.24) 

To  now  show  that  the  DO  expansion  can  reduce  to  the  PC  expansion,  we  start  from  the  DO 
field  equations  (3.12)-(3.14)  but  we  restrict  them  with  (3.23).  Then  the  equation  for  the 
stochastic  coefficients  {Yj  (t;u;)}^=1  is  not  used  since  those  are  chosen  a  priori  in  a  classic 
PC  equation.  Then,  our  equation  for  the  mean  field  (3.13)  provides  directly  the  equation 
in  the  set  (3.24)  that  corresponds  to  (C  (w))  being  the  constant  polynomial.  Finally,  to 
obtain  the  remaining  equations  in  (3.24),  we  start  with  the  third  of  the  DO  field  equations 
(3.14)  in  the  form 

•  ( x  f} 

CX(«)D(t) — lgt  '  =  Eu  [C  [u  (x,  t;  u)  ■  u\  Yj  (t;w)]  -  nVs  [E?  [C  [u  (x,  t;  ui) ;  u]  Yj  (t;w)]] . 

(3.25) 
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But,  from  (3.17),  we  have  Eu  [(£  [u  (x,  f;  u>) ;  uj\  ,  u/,.  (#,  t)}  Yj  (f;  a;)]  =  C dYk(t)v  =  0  since 

dt  V) W 

the  stochastic  characteristics  of  (£  (a;))  do  not  change  with  time.  Therefore,  we  have 
ITvs  [E^  [C  [u  (x,  t;  uj)  ;  a;]  (t;  a;)]]  =  0  in  (3.25).  Hence,  equation  (3.25)  and  the  mean 

equation  (3.13)  reduce  to  the  family  of  PC  equations  (3.24). 

3.5.2  Proper  orthogonal  decomposition 

The  POD  method  uses  a  priori  chosen  fields  {uj  (x)}^=1 ,  generated  either  from  an  ensemble 
of  experiments  or  from  direct  numerical  simulations  ( [59] )  and  provides  equations  either  for 
the  stochastic  or  the  deterministic  coefficients  {Yj  (i;  to)}j=1-  In  what  follows  we  show 
how  our  DO  equations  reduce  to  the  standard  POD  method  (given  the  standard  POD 
assumptions)  for  the  stochastic  case  since  the  deterministic  equations  follow  as  a  special 
case. 

In  a  standard  POD  method,  one  chooses  an  expansion  having  the  form 

S 

u  (x,  t;  u)  =  (t;  u)  u*  (x)  (3.26) 

i= 1 

where  {u^  (x)}^=1  are  fixed,  orthonormal  fields  and  {Xj  (f;cj)}^=1  are  stochastic  processes 
(in  general  with  non-zero  mean).  The  reduced  order  POD  evolutions  equations  are  then 
usually  obtained  by  Galerkin  projection  of  the  original  SPDE  (3.5)  onto  the  orthonormal 
fields  Uj  (x).  In  this  way  we  obtain  the  set  of  equations 

dX'^^  =  (£  [u  (x,  t;  u) ;  u] ,  uj  (x,  t))  .  (3.27) 

To  now  show  that  the  DO  expansion  can  reduce  to  the  POD  method,  we  start  from  Theorem 
1  and  the  DO  field  equations  but  we  restrict  them  with  (3.26).  We  consider  just  equations 
(3.12)  and  (3.13)  since  {u^  (x)}s=1  have  already  be  imposed  from  the  POD  method.  More¬ 
over,  we  note  that  the  coefficients  {Xj  (i;w)}*=1  of  the  POD  method  are  connected  to  the 
stochastic  coefficients  {Yj  (t;  u)}j=1  of  Theorem  1  through  the  relation 

Xj  (t;  u)  =  Yj  (t;  u)  +  (u  (x,  t) ,  u j  (x))  . 

Then,  by  differentiating  Xj  (t; u>)  and  using  equations  (3.12),  (3.13)  we  recover  equation 
(3.27) 

dXj  (t;  u)  _  dYj  (t;  u)  ,  /  du  (x,  t)  (  ^  \ 

dt  ~  dt  +\  dt 

=  (£  [u  (#,  t;  u) ;  u>\  -  Eu  [C  [u  (#,  t;  u) ;  w]] ,  (#,  t)) 

+  {. E “  [C  [u  (x,  t;  u>) ;  w]] ,  u j  (x,  t)) 

=  {£  [u  (•,  t;  oj)  ;u]  ,Uj  (#,t)) . 
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Chapter  4 

Evolution  of  the  stochastic 
dimensionality 


Abstract 

The  scope  of  this  chapter  is  to  develop  adaptive  criteria  for  the  dimensionality  of  the  of 
the  stochastic  subspace  in  the  context  of  the  dynamically  orthogonal  (DO)  field  equations. 
In  the  first  section  we  discuss  the  cost  scaling  of  the  DO  field  equations  with  respect  to 
the  number  of  stochastic  dimensions  used.  Subsequently,  we  present  adaptive  criteria  for 
the  contraction  and  expansion  of  the  stochastic  subspace  and  we  also  illustrate  how  the 
new  stochastic  dimensions  should  be  chosen  (when  the  stochastic  subspace  should  be  ex¬ 
panded)  according  to  stability  arguments.  Finally,  we  also  discuss  the  issue  of  updating 
both  the  stochastic  subspace  and  the  probabilistic  information  (i.e.  the  stochastic  coeffi¬ 
cients)  through  the  usage  of  full-field  data  if  those  are  available.  The  material  presented  in 
this  chapter  is  partially  contained  in  Sapsis  and  Lermusiaux,  2009,  2010  [129],  [126]. 


4.1  Introduction 

In  Chapter  2  we  saw  how  we  can  approximate  the  full  probability  measure  by  its  projection 
to  a  finite  dimensional  subspace,  the  stochastic  subspace  or  error  subspace  of  the  problem. 
Using  this  property  we  derived  an  exact  closed  set  of  field  equations  that  evolve  the  proba¬ 
bilistic  structure  for  fixed  stochastic  dimensionality  s.  In  this  chapter  we  will  discuss  how 
this  stochastic  dimensionality  should  vary  with  respect  to  time  according  to  the  correct 
response  of  the  system.  More  formally  we  may  think  of  the  stochastic  dimensionality  as 
the  minimal  number  of  latent  variables  needed  to  approximate  satisfactory  the  system  at  a 
given  time  instant. 

The  estimation  of  the  number  of  latent  variables  is  an  essential  step  in  the  process  of 
reduced-order  modeling  for  stochastic  systems  since  most  order-reduction  methods  need 
that  number  as  an  external  and  a  priori  user-defined  parameter.  However,  there  are  many 
situations  where  the  dimensionality  of  the  stochastic  subspace  needs  to  be  adapted  as  the 
system  evolves.  A  typical  example  is  a  stochastic  system  initiated  with  deterministic  ini¬ 
tial  conditions  where  the  stochastic  subspace  is  initially  an  empty  set  and  as  time  evolves 
acquires  non-zero  dimensionality.  Another  commonly  encountered  problem  where  it  is  es¬ 
sential  to  adapt  the  stochastic  dimensionality,  is  a  stochastic  system  exhibiting  transient 
dynamics  either  due  to  time  varying  external  excitation  or  inherent  instabilities. 
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Typical  applications  of  this  kind  are  atmospheric  and  oceanic  systems  where  the  strongly 
transient  and  non-stationary  character  of  the  dynamics  requires  adaptive  modeling  of  the 
dimensionality.  For  oceanic  data  assimilation  applications  a  convergence  criterion  based 
on  the  second-order  statistical  characteristics  of  the  system  state  is  defined  and  utilized  to 
control  and  adapt  the  stochastic  dimensionality  (see  [81],  [74],  [75],  [76],  [78]). 

Other  estimators  of  the  stochastic  dimension  come  from  fractal  geometry  including  the 
capacity  dimension  [112]  and  the  correlation  dimension  [53].  Other  methods  that  are  not 
primarily  intended  to  compute  the  fractal  dimension  can  also  be  used  to  evaluate  the  di¬ 
mensionality  of  the  stochastic  subspace.  One  of  the  most  commonly  used  is  the  Principal 
Component  Analysis  (PCA)  [71]  mainly  because  of  its  simplicity.  PCA  is  a  linear  method, 
meaning  that  the  estimator  cannot  identify  nonlinear  dependencies  but  only  gives  a  global 
dimensionality  of  an  object.  However,  as  we  illustrated  in  Chapter  2  this  can  be  sufficient 
information  since  the  nonlinear  dependencies  may  be  expressed  through  the  joint  probability 
density  function  defined  in  the  reduced  order,  time-varying,  space. 


4.2  Cost  scaling  with  the  stochastic  dimensionality 

The  most  important  obstacle  towards  the  numerical  solution  of  stochastic  dynamical  sys¬ 
tems  and  especially  of  high  dimensionality  such  as  SPDEs  is  the  exponential  growth  of  the 
number  of  unknowns  with  respect  to  the  stochastic  dimensionality  of  the  problem.  This  is 
also  known  as  curse  of  dimensionality  [14]  and  refers  to  the  exponential  growth  of  hyper¬ 
volume  as  a  function  of  dimensionality.  Therefore  in  a  direct  simulation  approach  where  N 
degrees  of  freedom  are  involved  in  every  stochastic  dimension,  the  storage  cost  of  having  s 
stochastic  dimensions  will  be  given  by 


Sdirect  (AO  =  O  (Ns)  . 

Hence,  for  high  or  infinite-dimensional  systems,  the  storage,  and  thus,  the  computational 
cost  is  prohibited.  In  the  Polynomial-Chaos  method,  where  the  stochastic  element  is  pro¬ 
jected  into  a  given  set  of  random  functions  the  storage  cost  grows  polynomially  as  (see  e.g. 
[162]) 

Spc  (AO  =  A^ri  ll  (a  +  r)  ~  O  ( Ns p) , 

q= 1  r= 0 

where  p  is  the  order  of  the  polynomial-chaos  approximation  and  s  the  stochastic  dimension¬ 
ality  of  the  problem.  Therefore,  the  growth  rate  strongly  depends  on  the  order  p  which  may 
need  to  be  sufficiently  large  in  order  to  capture  adequately  complex  statistical  responses. 
For  the  same  method,  the  associated  computational  cost  for  evolving  the  degrees  of  freedom 
depends  on  the  order  of  the  non-linearity,  q  ,  characterizing  the  evolution  equation  (e.g.  for 
Navier-Stokes  q  =  2)  and  it  will  scale  as 

Cpc  (N)  ~  O  ([ATsp]9) . 

where  the  exponent  q  comes  from  the  fact  that  for  a  q— order  polynomial  term  in  the 
system  equation  and  a  representation  consisting  of  Nsp  terms  the  number  of  multiplications 
required  for  the  computation  of  this  term  will  be  [Nsp]q  . 

In  the  ESSE  approach  the  storage  cost  grows  linearly  with  respect  to  the  stochastic 
dimension  s.  Since  the  evolution  of  the  probabilistic  information  is  done  through  Monte- 
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Carlo  simulation  the  computational  cost  does  not  depend  on  the  stochastic  dimensionality 
but  rather  on  the  number  of  Monte-Carlo  samples  that  will  allow  us  to  get  a  satisfactory 
approximation  of  the  stochastic  response. 

In  the  case  of  the  DO  field  equations  the  storage  cost  grows  linearly  with  respect  to  the 
stochastic  dimension  s,  while  the  computational  cost  grows  polynomially  with  an  exponent 
that  does  not  depend  on  s  but  rather  on  the  nonlinearity  q  associated  with  the  governing 
equations.  More  specifically,  as  we  presented  in  the  previous  section,  the  representation 
(3.10)  consists  of  s  +  1  deterministic  fields,  and  a  stochastic  process  that  takes  values  in  Ms. 
Even  though  the  stochastic  process  Y  (i;  lu)  carries  the  curse  of  dimensionality,  the  small  to 
moderate  size  of  s  (even  for  realistic  oceanic  applications  s  ~  O  (10)  —  O  (lO3)  is  sufficient, 
[82] )  allows  for  storage  of  the  probabilistic  structure  through  a  fixed  number  of  samples  (see 
Appendix  B).  Therefore,  the  main  storage  cost  comes  from  the  s  +  1  time  dependent  fields. 
Thus, 

SDO  (N)  =  O  (Ns) , 

Consequently,  for  a  system  operator  C  having  polynomial  nonlinearities  of  maximum  order 
q  the  computational  cost  will  be  given  by 

CDO  (N)  =  0([Ns]q). 

where  the  computational  cost  follows  from  similar  arguments  as  in  the  case  of  the  PC 
method. 

From  the  above  discussion  we  conclude  that  in  the  DO  methodology  the  storage  cost 
grows  linearly  independently  from  the  complexity  of  the  stochastic  solution  or  the  nonlin¬ 
earity  of  the  system  operator,  while  the  number  of  numerical  operations  grows  polynomially 
with  an  exponent  that  depends  exclusively  on  the  order  of  nonlinearity  of  the  system  oper¬ 
ator. 

In  Figure  4-1  we  present  the  computational  cost  in  seconds  with  respect  to  the  number 
of  modes  for  a  dynamical  system  described  by  Navier-Stokes  equations.  More  specifically, 
we  consider  the  stochastic  lid-driven  cavity  flow  which  is  described  in  detail  in  Chapter  5. 
The  red  curve  indicates  the  measured  time  for  a  particular  run  while  the  blue  line  is  the  best 
linear  fit.  The  inclination  of  the  best  linear  fit  is  equal  to  1.986  and  compares  satisfactory 
with  the  theoretical  prediction  i.e.  the  order  of  nonlinearity  which  for  Navier-Stokes  is  2. 


4.3  Update  of  the  stochastic  subspace  using  stability  prop¬ 
erties  of  the  SPDE 


In  this  section  we  will  study  criteria  for  the  dimensionality  selection  of  the  stochastic  sub¬ 
space  V5  as  the  system  evolves.  The  proposed  conditions  for  the  contraction  and  ex¬ 
pansion  of  the  stochastic  subspace  will  be  based  on  the  covariance  matrix  C Yi(t)Yj(t)  = 
Ew  [Yt  (t]Lo)  Yj  (t;cu)],  i.e.  on  the  second  order  characteristics  of  the  stochastic  field.  Note 
that  C Yi(t)Yj{t)  provides  information  about  both  the  intensity  of  the  uncertainty  that  charac¬ 
terizes  a  stochastic  field  but  also  the  principal  directions  in  H  over  which  this  stochasticity 
is  distributed. 
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Figure  4-1:  Computational  time  (sec)  for  the  lid-driven  cavity  flow  described  in  Chapter  5, 
using  different  number  of  modes  (red  curve).  The  blue  line  indicates  the  linear  ’best  fit’  in 
the  log-log  plot  and  it  has  an  inclination  equal  to  1.986  (2  is  the  theoretical  prediction). 


4.3.1  Conditions  for  the  evolution  of  the  stochastic  dimensionality 

For  the  covariance  matrix  C Yi(t)Yj(t)  we  have  the  set  of  eigenvalues  pj  (t) ,  j  =  1, ...,  s  and 
the  corresponding  eigenvectors  (j)j  (t) ,  j  =  1, ...,  s,  given  by  the  eigenvalue  problem 

GYi{t)Yj{t)<t>kj  (t)  =  p\  (t)  (/>ki  (t)  ■ 

To  relate  the  above  eigenvalues  and  eigenvectors  with  those  of  the  covariance  operator 
Cu(-,t)u(-,t)  (x,y)  we  observe  by  definition  of  our  representation  of  u  (x,  t]Lu)  (having  order 
s)  that 


cu(-,t)u(-,t)  (x,  y)  =  Eu  Yi  (t;  u)  Yj  (t;  u)  u*  (x,  t)  Uj  (y,  t)J 
=  (x,  t)  u j  (y,  t)T 


Then  we  can  easily  check  that  the  eigenvalue  problem 


j  cu(-,t)u(-,t)  (x^  y)  ui  (x>  t)  dx  =A?  (t)  Uj  (y,  t),  y  eD 

D 

has  s  non-zero  eigenvalues  given  by  pj  (t) ,  j  =  1 , . . . ,  s  with  the  corresponding  eigenfields 
given  by 

Vj  (x,i)  =  fa  (t)  u *  (x,  t)  ,  j  =  1, ...,  s 
where  cj)ij  (t)  is  the  i  element  of  the  j  eigenvector  <f)j  (t)  . 
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In  what  follows,  we  provide  one  condition  for  the  decrease  and  one  for  the  increase  of 
the  size  of  the  stochastic  subspace,  considering  arbitrary  contraction  time  tc  and  expansion 
time  te  at  which  this  can  happen. 

Condition  12  (Contraction  of  Vg)  The  stochastic  dimension  s  =  dim  Vs  will  be  de¬ 
creased  by  one  when  at  t  =  tc  the  minimum  eigenvalue  becomes  less  than  a  pre-defined 
critical  value  a2r 

mmpf  {tc)  <  a2r. 

I 

In  this  way  we  set  a  threshold  of  variance  below  which  uncertainty  is  sufficiently  small 
to  be  neglected.  The  stochastic  subspace  basis  elements  u,  (x,  tc)  as  well  as  the  stochastic 
coefficients  Yi(tc',oS)  have  to  be  updated  accordingly.  We  assume  that  {p2s  (tc)  ,  vs  (x,tc)) 
is  the  eigenpair  that  we  neglect  because  p2  (tc)  <  a2r.  Moreover,  we  denote  as  (x,  tc) 
and  Yf+  (fc;  u) ,  i  =  1, s  —  1  the  basis  elements  of  the  stochastic  subspace  V^_1  and  the 
corresponding  stochastic  coefficients  respectively  after  the  application  of  the  contraction 
criterion.  By  choosing  the  basis  elements  u)1"  (x,  tc)  to  be  identical  with  the  eigenfields 
Vi  (x,fc) ,  *  =  1, ...,  s  —  lwe  have  the  stochastic  subspace  V^_1  which  is  contracted  relative 
to  Vs  exactly  by  the  eigendirection  vs  (x,fc)  that  corresponds  to  the  minimum  eigenvalue 
(note  that  orthonormality  of  u,  (x,ic) ,  i  =  1,  ...,s  —  1  is  preserved).  Then,  the  state  of  the 
system  at  t  =  tc  will  be  described  by 

S—  1 

u+  (x,  fc;  u)  =  u  (x,  tc )  +  ( tc ;  u)  Vi  (x,  tc) ,  uj  G 

1=1 

where  the  stochastic  coefficients  can  be  easily  found  by  projecting  the  stochastic  part  of  the 
solution  u  (x,  tc ;  u)  —  u  (x,  tc)  to  the  basis  Vj  (x,fc) ,  j  =  1, s  —  1 

Y+  (tc,  uj)  =  4>ij  (tc)  Yi  (tc,  u),  i  =  1, ...,  s  -  1 

Condition  13  (Expansion  of  Vs)  The  stochastic  dimension  s  =  dim  Vs  will  be  in¬ 
creased  by  one  when  at  t  =  te  the  minimum  eigenvalue  becomes  greater  than  a  pre-defined 
critical  value  Y2  >  a2. 

mm  pi  (te)  >  Ycr  >  acr. 

I 

The  additional  stochastic  dimension  is  chosen  for  simplicity  to  have  a  stochastic  coef¬ 
ficient  ( te  ',u)  that  is  normally  distributed  with  variance  a2r.  Moreover,  it  is  assumed 

statistically  independent  from  the  existing  stochastic  coefficients.  Both  of  the  above  assump¬ 
tions  are  based  on  the  fact  that  the  stochastic  intensity  along  the  new  direction  is  small  ( a2r ). 
However,  the  selection  of  the  the  additional  basis  field  uj+1  (x,  te)  is  not  straightforward 
and  will  be  made  based  on  stability  arguments  of  the  system  operator  C. 

4.3.2  Selection  of  new  stochastic  dimensions 

We  will  now  describe  the  directions  in  H  which  are  not  included  in  the  stochastic  subspace 
V g  and  which  have  the  larger  tendency  to  grow  (most  unstable)  in  terms  of  the  variance. 
In  what  follows  we  will  assume  that  uncertainty  is  small  and  uniform  in  the  orthogonal 
complement  of  the  stochastic  subspace  (the  one  that  until  now  is  not  considered  stochastic) . 
Based  on  this  assumption  we  may  chose  the  new  direction  based  only  on  the  largest  growth 
rate  (see  e.g.  [73],  [63]). 
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First  we  give  some  definitions  that  will  be  essential  to  our  analysis. 


Suppose  $  (u)  :  H  — >  H  is  an  operator  from  the  space  of  square  integrable  stochastic 
fields  H  to  itself.  The  operator  $  will  be  called  Frechet  differentiable  (see  e.g.  [26])  if  for 
any  u  £  H  there  exists  a  bounded,  linear  operator  [h]  :  H  — >  H  such  that  the  following 
limit  exists 


lim 


$  (u  +  eh)  -  4>  (u)  -  s-^f-  [h] 

£ 


=  0. 


where  1 1 u 1 1  ^  =  (u,  u)  is  the  norm  induced  by  the  inner  product  of  the  Hilbert  space  H.  In 
this  case  we  will  have  (Cartan,  1971  [26]) 


$  (u  +  eh) 


$(u) 


(IT  (u) 
<Ju 


[h  ]  +  0{e2). 


(4.1) 


In  what  follows  we  will  study  the  normal  stability  of  Vs,  i.e.  the  stability  of  the 
reduced  system  to  perturbations  which  are  normal  to  Vg.  To  this  end  we  will  use  Normal 
Infinitesimal  Lyapunov  Exponents  (NILE)  that  have  been  used  in  the  study  of  normal 
stability  properties  of  invariant  manifolds  ([57]).  More  specifically,  we  consider  a  small 
perturbation  of  an  element  in  V5  at  the  time  instant  te  that  has  the  form 


U  (x,  te\ Lo)  =  eT  {te,  Lo)  d  (x,  te)  +  U  (x,  te\  Lo)  . 

where  u  £  Vs,  e  is  a  small  real  number,  1?  (x,  te)  is  a  deterministic  field  that  is  normal  to 
the  stochastic  subspace  Vs,  and  T  (te',uj)  is  a  square-integrable  random  variable  that  is 
independent  from  the  stochastic  processes  L)  (L  u>) ,  i  =  1, ...,  s,  t  <  te.  Then  from  the  s  +  1 
dimensional  DO  equations  we  will  have 

£~dt~ ~  =  ^  ^  (*’ t;  ^  ^  ~  EU  ^  ^  **’ t]  ^  ^  (*’  *)) 

Then,  by  expanding  C  [u  (•,  t;  lo)  ;  c o\  around  u  using  equation  (4.1)  we  obtain 

C  [u  (x,  t-,w)-,u>\  =  C  [u  (x,  t ;  u)]u\  +  eT  (t ;  lo)  —  ^  ^  ^  [t?  (x,  t)]  +  O  {e2) 

Moreover,  since  T  {t;  lo)  is  zero-mean  we  will  have  for  the  limit  t  — ►  te 

lim  Eu  [C  [u  (x,  t;  uo) ;  a;]]  =  lim  Eu  [C  [u  (x,  t;  lo)  ;  a;]]  +  O  (e2) 

t  t  ^tg 


Now  inserting  these  latter  two  equations  in  the  equation  for  T  (t;  lo)  we  have  for  t  — >  te 
_  dT  [t]  lo) 


dt 


=  {C  [u  (#,  te,  to)  ;  u\  -  E“  [C  [u  (•,  te,  to)  ;  w]]  ,  I?  (•,  te)) 


t=te 


+  eT  (te;w)  (*^e,^)  ,u]  (#j  te)] , $  (.,  te)^  +  O  (e2) 

Then  we  multiply  with  2T  (t;  lo)  and  apply  the  mean  value  operator  to  obtain  for  the  limit 
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t  — ►  te,  £  — >  0 


dEN  [T2  (t;  w)' 


dt 


=  2E“  [T2  (ie;w)]  /  E 


t=te 


SC  [u(»,fe;  lu)  ;w] 
<5u 


Therefore  the  NILE  in  this  case  will  be  given  by 


ate  [u]  =  2  max  Q  [$]  =  2  max  (  Eu 

i?e\4x  tJeV,1 

M=i  ll*ll=i 


<!£[»(.  <;u,);u>] 
<5u 


,*(•,*)  }  (4.2) 


The  last  quantity  is  a  measure  of  the  maximum  potential  growth  of  perturbations  which  are 
not  contained  in  ~Vg.  Using  the  above  quantity  we  can  predict  based  on  the  current  state  of 
the  system  and  the  current  form  of  the  stochastic  subspace  Vs  which  perturbation  id  (x,  te) 
will  grow  faster  and  therefore  we  can  update  or  expand  the  stochastic  subspace  accordingly. 

Another  approach  that  is  commonly  used  in  the  weather  prediction  literature  is  based 
on  the  instantaneous  left  singular  vector  of  the  tangent  linear  model  considered  over  a  finite 
time  interval  extending  from  the  current  time  to  a  future  time  instant  (the  so  called  ‘linear 
regime’).  This  approach  is  based  on  the  assumption  of  linearized  dynamics  over  this  short 
time  interval  that  allows  to  find  the  perturbation  that  will  have  the  maximum  growth  over 
this  finite  interval  (See  e.g.  [83],  [73],  [63]). 


Numerical  computation  of  the  NILE 

In  order  to  compute  the  NILE  we  first  approximate  V^r  by  a  finite  base  {$*  (x,  .  This 

can  also  be  seen  as  the  first  iteration  of  the  breeding  procedure  ([63],  [81]);  this  ’first  step’ 
approach  is  also  utilized  in  ESSE  method  except  that  in  the  present  work  we  do  the  analysis 
in  continuous  time. 

Note  that  such  basis  can  always  be  constructed  using  any  finite  base  that  approximates 
the  space  of  square  integrable  deterministic  fields  L2  (e.g.  Fourier  modes)  and  then  applying 
Gram-Schmidt  process.  By  considering  such  a  base  we  will  have 

9  9 

id  (x,  te)  =  ^2  di-di  (x,  te)  with  y~]qf  =  1. 
i=  1  i=  1 

Then,  using  the  linearity  of  the  Frechet  derivative  with  respect  to  id  we  will  have 


Q[ti\  =  Q[ai,...,aq] 


=  CLiCLj  (  E ^ 

—  2  ( Qij  dr  Qji)  di dj 


SC  [u  (•,  t;  uj)  ;  uj] 


Su 


[0*  (•>*)] 


,idj  (#,f) 


where  Qij  is  a  q  x  q  matrix  whose  elements  are  given  by 


Qij  —  \E{ 


<!£[»(.  t;u,);  a,] 
Su 


,idj  (*,t) 


Note  that  in  the  equation  for  Q  [i?]  we  have  used  the  fact  that  for  any  matrix  Q  and  vector 
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x  we  have 

x7  Qx  =  (x7  Qx)2  =  x7  Q7  x  =  -x7  (Q  +  QT)  x 

Therefore  the  problem  has  been  reduced  to  the  optimization  of  the  quadratic  function  Q 
over  the  unit  sphere  a7  =  1.  Note  that  the  property  a ^  =  1  follows  from  the  assumption  of 
uniform  variance  for  all  the  modes  which  are  not  included  in  the  stochastic  subspace.  For 
the  case  where  we  had  some  information  for  the  size  of  the  various  perturbations  in  the 
orthogonal  complement  we  would  have  a  weighted  problem  according  to  the  variances  of 
the  individual  modes  in  the  orthogonal  space. 

The  last  optimization  problem  has  always  a  solution  since  the  unit  sphere  is  a  compact 
set.  Moreover,  since  Q  is  quadratic  we  will  have 

tul  =  A  max  [Qij  +  Qji] 

where  Amax  denotes  the  maximum  eigenvalue.  Based  on  the  above  analysis  we  choose 
the  new  direction  u++1  (x,  te)  in  the  expanded  stochastic  subspace  as  the  critical  direction 
'dc  (x,  t)  for  which  maximum  growth  of  variance  will  occur 

U++1  (x,  te)  =  tic  (x,  t )  =  ac,i$i  (x,  t) 

where  {ac,i}Qi=i  is  the  eigenvector  of  Qij  +  Qji  that  corresponds  to  the  maximum  eigenvalue 
Amax  [ Qij  T  Qji]  ■ 

4.4  Update  of  the  stochastic  subspace  using  data  and  mea¬ 
surements 

In  many  problems  modeled  through  the  stochastic  framework,  available  data  or  measure¬ 
ments  can  improve  significantly  the  accuracy  of  the  stochastic  solution.  The  scope  of  this 
section  is  to  describe  how  this  information  can  be  merged  with  the  numerically  evolved 
stochastic  fields  within  the  context  of  DO  equations. 

Generally  data  and  measurements  are  available  in  arbitrary  locations  in  the  domain  of 
interest.  The  optimal  estimation  of  gridded  fields  directly  from  the  spatially  irregular  and 
multivariate  data  sets  that  are  collected  by  varied  instruments  and  sampling  schemes  is  a 
problem  studied  in  the  context  of  objective  analysis  (see  e.g.  [160],  [1],  [2])  and  will  not  be 
studied  in  this  work.  For  schemes  that  utilize  raw  data  to  learn  the  dominant  (multivariate) 
stochastic  subspace,  we  refer  to  [73]  and  [75] .  Here  we  assume  that  data  or  measurements 
information  is  expressed  in  field  form  known  at  particular  time  instants.  Based  on  this 
assumption  we  first  define  some  essential  notation  for  the  analysis  that  will  follow. 


4.4.1  Data  and  measurements  formulation 


We  denote  the  time  instant  where  data  is  available  as  tQ ,  the  unbiased  estimated  field 
(through  objective  analysis)  as  u(x,  ta),  and  the  covariance  operator  for  the  associated 
error  £  (x,  tQ\  lo)  =  u  (x,  tQ)  —  u  (x,  lu)  as 


Ces  (x,  y)  =  Eu  £  (x,fG;u;)  £  (y,f0;w)J 


58 


Figure  4-2:  Decomposition  of  the  stochastic  subspace  Vs  based  on  the  data  subspace  Vo- 


For  simplicity  we  also  assume  zero-mean  Gaussian  statistics  for  the  estimated  field.  To 
express  the  available  information  in  the  DO  framework  we  perform  a  spectrum  analysis  of 
the  covariance  operator  by  solving  the  following  eigenvalue  problem 

J Css  (x,  y)  vs,i  (x)  dx  =A ltivs,i  (y ) ,  y  €D.  (4.3) 

D 

from  which  we  obtain  a  set  of  eigenpairs  ^A|  vs^i  (x)^  i  =  1,2,....  Then,  based  on  the 

critical  variance  threshold  a2r  (below  which  stochasticity  is  negligible),  we  obtain  the  full- 
field  data  subspace  defined  as 

Vo  =  span  { v£,i  (x)  |A§ ^  >  a2cr  } 

defined  as  the  span  of  all  eigenfields  vs.i  (x)  associated  with  important  variance  (A |  -  >  a2r). 


4.4.2  Update  of  the  stochastic  information  inside  V5 

To  update  the  stochastic  information  of  the  current  state  of  the  system  we  partition  the 
stochastic  subspace  V5  into  two  orthogonal  (i.e.  disjoint)  linear  subspaces  as  follows 

v5  =  vsnv0®vsnv^ 

where  ®  denotes  the  direct  sum  of  the  two  subspaces  (Figure  4-1).  Note  that  along  the 
dimensions  contained  in  the  subspace  V^nV^  C  V5,  the  available  information  from  data 
and  measurements  guarantees  accurate  estimation  since  the  associated  data  variance  is  less 
than  a^r.  Therefore,  for  these  directions,  we  may  neglect  those  stochastic  dimensions  and 
update  the  stochastic  subspace  and  the  mean  directly  as 

Vs— >  VsHVo,  and  s— >  dim  Vs  fl  Vo 
u  (x,  t0)  -»•  u  (x,  t0)  -  nVsnVx  [u  (x,  to)]  +  nvsnv^  [*  (x>  Q] 
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where  for  the  stochastic  subspace  we  maintain  only  the  directions  lying  on  the  intersection 
of  the  stochastic  subspace  and  the  data  information.  For  the  mean  we  substitute  completely 
the  information  computed  through  the  evolution  equations  with  the  measured  information 
for  which  there  is  good  accuracy,  i.e.  the  information  corresponding  to  eigendirections  with 
A|  j  <  a^r  which  are  contained  in  Vq. 

The  next  step  of  our  analysis  involves  the  update  of  the  remaining  stochastic  coefficients 
Yi  (t0-,  to)  that  describe  the  probabilistic  structure  in  the  reduced-dimension  stochastic  sub¬ 
space  Vs.  We  have  by  definition  of  £ 

u  (x,  t0)  =  £  (x,  t0 ;  to)  +  u  (x,  t0)  +  Yi  (ta;  tv)  u,  (x,  t0) 

By  projecting  the  above  equation  to  every  basis  element  of  the  stochastic  subspace  we  obtain 
for  every  k  =  1, ...,  s 

(u  (#,  to)  ,  Uk  (•,  to))  =  (£  (•,  to]  to)  ,  Uk  (#,  to))  +  (u  (#,  to)  ,  Uk  (#,  to))  +  Yk  (to,  to)  . 

We  shall  now  do  a  Bayes  ‘data  assimilation’  update  in  the  Y  space.  Using  the  above  relation 
we  can  apply  Bayes  rule  to  update  the  probability  density  function  describing  the  stochastic 
coefficients.  More,  specifically  we  will  have 

f  ({(u(»,t0)  ,uk  (•,t0))}sk=1 1  y)  /y  (yytp 
f  /({(u  (•,  io) ,  ufc  (•,  to))}fc=1 1  z)/y  (Mo)  dz 

Rs 


/y  (yUo|  {(u(»,t0),uk  (•,t0))}sk= i) 


where  y  is  the  argument  for  the  random  variable  Y  (tD;  to)  and  fy  is  the  corresponding 
probability  density  function  (see  Chapter  2  -  Section  2.2).  We  have  assumed  Gaussian 
statistics  for  the  error  field  £  (x,  t0\  to)  .  Therefore, 

/  ({(u  (•,  to) ,  W  (•,  t0))}sk= i  |  y)  =  Af((u  (•,  t0) ,  uk  (#,  to))  +  yk,  &ij) 


where  H,?  is  the  covariance  matrix  defined  as 


uf  (xi ,to)  Cg£  (xi,x2)  u j  (x2 ,  t0)  dx idx2,  i,j  =  1, 


D  D 


Thus,  the  pdf  describing  the  updated  stochastic  coefficients  will  be 


/y  (y,  tQ\  {(u  (#,  t0) ,  ufc  (•,  t0))}sk= i) 


Af((u(»,t0)  ,uk  (»,t0))  +  yk,3 jj)  fY  (Mo) 

/  J\f({u(»,t0)  ,uk  ( »,t0 ))  +  yk,S ij)  /Y  (z ,t0)  dz 


To  be  consistent  with  the  DO  formulation  we  finally  need  to  center  the  above  density,  so 
that  the  updated  Yk  ( t0',to )  are  zero  mean  (Figure  4-2).  Specifically,  we  will  have 

u  (x,  t0)  ->  u  (x,  t0)  +  mkuk  (x,  t0) ,  with  mk  =  ykfy(y,t0\  {(u  (•,  tQ)  ,.uk  (#,  i0))}^=1)  dy 

Rs 

/y  (y ,to)  ->•  /y  (y  +  m,t0|  {(u(»,io)  ,ufe(«,i0))}fe=1) . 
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4.4.3  Expansion  of  the  stochastic  subspace  V5 

The  second  stage  involves  the  consideration  of  the  stochastic  dimensions  of  Vo  which  are 
not  included  into  the  stochastic  subspace  V,§.  More  specifically,  we  consider  the  space 
Wo  =  Vo  n  Vg.  This  linear  subspace  contains  directions  with  important  uncertainty 
according  to  the  estimation  procedure  and  hence  these  should  be  included  into  the  stochastic 
subspace  V5.  This  can  be  done  in  more  than  one  ways.  One  approach  is  the  expansion  of 
the  stochastic  subspace  V5  using  all  the  new  dimensions  contained  in  Wo-  However,  this 
method  may  involve  significant  cost.  Another  approach  (ESSE  learning)  is  to  add  only  a 
single  dimension  for  every  synoptic  batch  of  data  [75]. 

A  more  efficient  alternative  is  to  combine  the  information  for  Wo  with  the  analytical 
arguments  presented  in  the  previous  section  in  order  to  obtain  the  direction(s)  in  Wo  that 
tend  to  obtain  larger  values  of  variance  according  to  the  dynamics  of  the  system.  In  this 
way,  we  may  enhance  V5  only  with  the  most  unstable  directions  of  the  space  Wo¬ 
lf  we  select  only  the  most  unstable  direction  in  Wo,  then  according  to  the  results  of 
the  last  section  this  will  be  given  by 


dim  Wo 

Us+1  (x,  to)  =  ^  diWi  (x) 

l—  1 


where  w,  (x)  is  a  basis  that  spans  Wo  and  is  the  eigenvector  associated  with  the 

maximum  eigenvalue  of  the  matrix  Qij  +  Q3% ,  with 


Qij  =  (  E' 


5C  [u(«,t0;o;)  ;cu] 
8u 


K  (•)] 


,  W j  (« 


Depending  on  the  nature  and  scale  of  the  problem  and  the  size  of  the  stochastic  subspace 
a  combination  of  the  two  approaches,  i.e.  adding  those  directions  in  Wo  with  important 
variance  as  well  as  those  which  are  most  unstable,  may  result  in  more  efficient  results.  Fi¬ 
nally,  the  corresponding  stochastic  coefficients  Ys+\  (fc;  lo)  will  follow  Gaussian  distribution 
with  variance 


D  D 


(xi,  to)  C££  (xi,  x2)  us+i  (x2 ,  to)  dxidx2 


and  mean 

E“  [Ys+i  ( t0\ cu)]  =  (u  (#,  t0) ,  us+i  (#,  t0)) 

As  we  mentioned  in  the  previous  section  appropriate  modification  of  the  mean  field  should 
be  made  so  that  Yg+i  (t0',u))  is  centered  according  to  the  DO  formulation. 
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fr  (M  ) 


Jw  «w.  it, )+  zt ; s )./,  (2* , tM  )dz 


y 


Figure  4-3:  Update  of  the  probability  density  function  ,/V  (y ,  t)  using  data  or  measurements. 
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Chapter  5 


Application  of  dynamically 
orthogonal  field  equations  to 
random  fluid  flows 


Abstract 

In  this  chapter  we  will  apply  the  Dynamically  Orthogonal  field  equations  for  the  case  of 
two  dimensional  random  flows  described  by  Navier-Stokes  equations  including  the  Coriolis 
force.  The  Navier-Stokes  equations  contain  quadratic  nonlinearities,  which  as  we  will  see, 
are  responsible  for  the  transfer  of  energy  from  the  mean  flow  to  the  various  stochastic 
DO  modes  in  the  form  of  variance.  The  quadratic  terms  are  also  the  cause  of  transfer 
of  uncertainty  among  different  DO  modes.  In  what  follows  we  will  illustrate  the  above 
mechanisms  through  the  consideration  of  special  cases  involving  externally  forced,  forced 
through  the  boundary,  and  free  systems.  More  specifically,  in  the  first  two  sections  we  will 
formulate  the  problem  and  we  will  derive  closed,  evolution  equations  for  the  mean  field,  the 
scalar  stochastic  coefficients,  and  the  DO  modes.  Note  that  the  Navier-Stokes  equations 
involve  a  diagnostic  variable,  the  pressure,  which  is  not  described  explicitly  and  hence  special 
treatment  is  needed.  In  the  third  section  we  will  discuss  the  case  of  stochastic  boundary 
conditions  and  we  will  prove  that  this  family  of  problems  can  always  be  reformulated  as 
problems  with  deterministic  boundary  conditions  and  suitable  random  forcing.  In  Section 
4  we  will  apply  the  methodology  developed  in  Chapter  4  to  express  the  most  unstable  mode 
and  in  Section  5  we  will  derive  expressions  for  the  transfer  of  energy  from  the  mean  flow  to 
and  among  the  stochastic  modes.  In  the  last  three  sections  of  the  chapter  we  will  present 
numerical  results  for  specific  geometries  and  forcing  configurations  and  we  will  also  examine 
convergence  properties  of  the  proposed  methodology.  The  details  for  the  numerical  solution 
of  the  derived  equations  are  included  in  Appendix  B.  Part  of  the  material  presented  in  the 
current  chapter  is  contained  in  Sapsis  and  Lermusiaux,  2009  [129]. 


5.1  Formulation 

We  will  now  formulate  the  DO  field  equations  for  the  special  case  of  fluid  flows  governed 
by  stochastic  Navier-Stokes.  The  general  equations,  for  an  incompressible,  two  dimensional 
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fluid  in  a  domain  D  in  a  rotating  frame  at  frequency  /  have  the  form 


—  =  —  Vp  +  —  Au  —  u.Vu  —  /k  x  u  +  r  (x,  t)  +  p  (x,  t\u)  =  C  [u  (x,  t;  id)  ;  u\ 
ot  Re 

0  =  div  u 


where  u  =  (u  (x,  t\uS)  ,v  (x,  t\  id))  is  the  flow  velocity  field,  and  k  is  the  unit  vector  for  the 
2— direction.  The  pressure  field  is  denoted  with  p(x,t;u>),  f  =  fo  +  Poy  is  the  Coriolis 
coefficient  under  the  beta  plane  approximation,  r  (x,  t)  =  (tx  (x,  t) ,  ry  (x,  t ))  is  the  external 
deterministic  stress  acting  on  the  fluid,  and  p(x,t;id)  =  (px  (x,  i;  u) ,  py  (x,  t\  u))  is  the 
zero-mean  stochastic  component  of  the  stress  for  which  we  assume  known  the  complete 
probabilistic  information.  In  what  follows  we  will  use  the  DO  field  equations  derived  in 
Chapter  3  with  inner  product 


(ui,  u2)  =  J  {uiU2  +  V1V2)  dx. 


D 


We  consider  the  correlation  operator  for  the  stochastic  part  of  the  excitation  Cw  (x,  y)  = 
Eu  Lp  (x,  t]  u ;)  ip  (y ,  t;  lu)t  .  To  diagonalize  the  probability  measure  associated  with  p  (x,  t;  uj) 
we  solve  the  following  2D-vector  eigenvalue  problem 


j  Cw  (x,  y)  ipr  (y ,  t)  dy  =X2rpr  (x,  t) 

D 

In  this  way  we  obtain  the  principal  directions  over  which  the  probability  measure  is  spread 
in  the  variance  sense.  Retaining  only  the  the  first  R  terms  we  obtain  the  following  approx¬ 
imation  of  the  stochastic  field  cp  (x,  t;  uj) 

R 

ip  (x,  i;  w)  =  (t;  uj)  pr  (x,  t)  =  Zr  (t;  id)  pr  (x,  t) 

r= 1 

where  R  is  defined  by  the  order  of  truncation  of  the  full  series,  and  Zr  (t;  id)  are  the  stochastic 
coefficients  given  by 

Zr  (t]ld)  =  (p(-,t]id)  ,pr  (-,t))  . 

For  simplicity  we  assume  that  the  boundary  conditions  for  the  velocity  are  either  determin¬ 
istic  Neumann  or  Dirichlet 


u  (£,t;  id)  =  udDl  (£,t)  ,  ^ 

^  (£,t;  id)  =  hdL>2  (£,t)  ,  £  €dD2 

The  case  of  stochastic  boundary  conditions  will  be  discussed  Section  3. 


Furthermore,  we  assume  that  the  initial  conditions  are  stochastic  with  known  statistics 
given  by 


U  (x,  to!  w)  =  Uq  (x;  id)  ,  x  6D,  id  E  II 
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5.2  Derivation  of  DO  Navier-Stokes  equations 

We  will  first  calculate  the  stochastic  operator  C.  By  using  the  DO  representation 


u  (x,  t;ui)  =  u  (x,  t)  +  Yi  (t;  uj)  u*  (x,  t) 
into  Navier-Stokes  equations  we  obtain 

C  [u  (x,  f;  u ;) ;  w]  =  —  Vp  +  —  Au  —  u.Vu— /k  x  u  +  r  (x,  t) 

Re 


(5.2) 


Yi 


— Au*  -  Uj.Vu  -  u.Vu*  -  fkxui 
Re 


-  - YjYj  [uj.Vuj  +  Uj.Vuj]  +  Zr  (t;  uj)  <pr  (x,  t) 


Moreover,  by  inserting  the  DO  representation  in  the  continuity  equation  we  obtain 

div  u  -j-  Yi  (t;  uj)  div  Uj  =  0. 

But  since  Yt  (f ;  uj)  is  random  the  above  equation  has  the  equivalent  form 


div  u  =  0 

div  Uj  =  0  ,  i  =  1, s. 

A  very  important  property  of  Navier-Stokes  equations  that  allow  for  the  efficient  applica¬ 
bility  of  the  DO  formulation  is  the  analyticity  of  the  evolution  operator  C  with  respect 
to  the  argument  u  (x,  t;  uj)  .  This  property  allows  for  the  expression  of  the  operator  into  a 
polynomial  series  that  involve  the  unknown  quantities  of  the  DO  representation  (eq.  5.2). 
Through  this  property  we  are  able  to  derive  closed  and  exact  evolution  equations  whose 
right  hand  side  depends  from  moments  of  the  stochastic  coefficients,  the  DO  modes  and  the 
mean. 

Note,  that  for  the  case  of  a  non-smooth  operator  C  we  are  not  able  to  expand  into  a 
polynomial  series  therefore,  even  though  the  DO  equations  are  valid,  it  is  not  possible  to 
compute  their  right  hand  side  efficiently  using  moments  of  the  coefficients  Y).  Instead,  in  this 
case  we  need  moments  of  the  full  field  u  (x,  t;  uj)  which  even  though  they  are  available  (since 
the  DO  formulation  includes  the  full  probabilistic  information)  it  is  not  straightforward  to 
compute.  For  the  case  where  the  non-snroothness  of  the  operator  C  occurs  in  points  where 
u  (x,  t;  uj)  has  very  low  probability  to  exist  we  may  approximate  the  operator  by  a  polynomial 
series.  We  emphasize  that  the  above  described  technical  complication  characterize  in  the 
same  way  every  method  that  is  based  on  an  a  priori  representation  of  the  solution  (i.e.  POD 
or  PC).  Monte-Carlo  based  methods  are  not  sensitive  to  this  point. 

5.2.1  Stochastic  pressure  field 

To  derive  an  equation  for  the  pressure  we  need  to  understand  its  role  in  the  stochastic 
context  of  the  operator  C  given  above.  Pressure  (for  a  flow  with  constant  density)  is 
the  stochastic  quantity  which  guarantees  that  every  possible  realization  uj  the  evolved  field 
(u  (x,  t]uj),v  (x,  f;  a;))  is  divergence- free  (take  the  divergence  of  the  momentum  equation  5.2 
and  use  the  family  of  continuity  equations  to  show  this).  Therefore,  the  stochastic  pressure 
should  be  able  to  balance  all  the  non-divergent  contributions  from  the  terms  involved  in 
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the  operator  C  (equation  (5.2)).  To  this  end  we  choose  to  represent  the  stochastic  pressure 
field  as 

p  =  Po  +  Yj  (t;  uj)  pi  -  Yi  (t;  uj)  Yj  (t;  uj)  Pij  +  Zr  (t;  uj)  br 

Based  on  the  above  discussion,  the  mean  pressure  field  components  should  satisfy  the  fol¬ 
lowing  equation 

Apo  =  div  |-u.Vu-/k  x  u  +  r  (x,  t)^j 
with  boundary  conditions  often  given  by 

(£,t)  =  0,  £  edDi  and  po  (£,t)  =  const.  £  £dD2 

an 

Note,  that  following  common  practice  in  fluid  mechanics  literature  [28]  we  have  chosen 
homogeneous  Neumann  condition  for  the  pressure  on  the  boundary  dD\  and  piece-wise 
constant  Dirichlet  condition  for  the  boundary  dD2.  As  we  will  see,  this  property  is  key  for 
simplifying  the  evolution  equations  for  the  stochastic  coefficients  and  also  for  the  numerical 
implementation  using  projection  methods. 


The  stochastic  terms  in  C  multiplied  with  Yj  (t;  uj)  will  be  balanced  through  the  following 
equation 

A  pi  =  div  Uj.Vu  —  u.Vuj  —  /k  x  ,  i  =  1, ...,  s  (5-4) 

with  boundary  conditions 
dv  ■ 

-7^(^)=0,  £  £dDi  and  Pi(£,t)  =  0,  £,  £dD2,  i  =  l,...,s. 

where  the  Dirichlet  condition  is  taken  to  be  homogeneous  since  the  boundary  conditions  of 
the  full  problem  have  been  assumed  deterministic  (the  generalization  for  stochastic  bound¬ 
ary  conditions  is  presented  in  the  next  sections).  Similarly,  for  the  stochastic  terms  multi¬ 
plied  by  Yj  (t;  uj)  Yj  (t;  uj)  we  will  have 

APij  =  \  div  (uj.Vuj  +  Uj.Vuj) ,  i,  j  =  1, ...,  s 
and  boundary  conditions 

^-(£,*)  =  0,  ?  and  Pij  (£,t)  =  0,  £  edD-2,  i,j  =  l,...,s. 

Finally,  the  forcing  terms  will  be  balanced  through  the  family  of  equations 

Abr  =  div  (pr  (x,  t) ,  r  =  1, R. 


with  boundary  conditions 

^(£>*)  =  0,  £  &dDi  and  qr  (£,t)  =  0,  £  edD2,  r  =  l,...,R. 

The  above  set  of  equations  guarantees  that  for  every  realization  uj  the  evolved  field  (u  (x,  t]uj),v  (x,  t;  a>)) 


66 


will  always  be  incompressible.  Moreover,  the  evolution  operator  C  will  take  the  form 


C  [u  (x,  t;  l o)\uj\  =  —  Vpo  +  w—  Au  —  u.Vu— /k  x  u  +  r  (x,  t) 

Re 


Yi 


V pi  +  —  Au*  -  Uj.Vu  -  u.Vuj  -  /k  x  u; 
Re 


-YiYj 


-Vpij  +  ^Uj.Vuj  +  -^Uj.Vuj 


+  Zr  (t;  u)  [~Vbr  +  ipr  (x,  t)] 


5.2.2  Evolution  of  the  mean  field  u  (x,  t;u>) 


Using  the  corresponding  DO  equation  for  the  mean  we  obtain  the  set  of  deterministic  PDEs 

(9u  1 

—  =  — Vpo  +  v—  Au  —  u.Vu— /k  x  u  +  r  (x,  t)  (5.5a) 

ot  Re 

[  1  1  1 

-  c V;(t)YRt)  -Vpij  +  -Uj.Vuj  +  -Uj.Vt 

0  =  div  u 

with  the  following  boundary  conditions 

u  (£,f;  u>)  =  udDl  (£,t)  ,  £  €dDi 

du 

—  (£,f;  w)  =  h dr>2  (£,t) ,  £  GdD2 


(5.5b) 


5.2.3  Evolution  of  the  stochastic  subspace  basis  u*  (x,  t]u>) 

We  will  first  calculate  the  quantity  [£  [u  (x,  t;  w) ;  w]  (f;  w)].  We  will  have 


[£  [u  (x,  t;  w) ;  w]  YJ-  (i;  w)]  =  Cym(t)yi(t) 


V Pm  +  —  Aum  -  um.Vu  -  u.Vum  -  / k  x  u„ 
Re 


-  M 


yRt)ym(t)yn(t) 


-Vpmn  +  ^Um.Vu„  +  ^U„.VU, 


+  Cy.(t)Zr(t)  [—Vbr  +  (fr  (x,  f)] 

Multiplying  with  the  inverse  matrix  C Yi(t)Yj(t)  we  will  have  (defining  Q,;) 

Qi  =  Cy/^yRt)^  tu  (x’  ^  W)1 

-Vpi  +  ^-Auf  -  Uj.Vu  -  u.Vuj  -  fkxui 
Re 


CY i(t)Yj(t)MYj 
+  ^-'yi(t)yJ(t)^'UWzr(t)  [~V6r  +  <pr  (x,  f)] 


Wi(t)yRt)iV1U(t)ym(t)yn(t) 

i-i 


V Pmn  +  ^Um.Vun  +  ^U„.Vum 
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Note  that  C Yjt)Y  (t)  88  a  covariance  matrix  is  always  symmetric,  positive-definite  thus  in¬ 
vertible.  Using  the  last  expression  we  obtain  the  evolution  equations  for  the  basis  Uj  (x,  t;  u) 

—  Q i  (Qij  Um)  Um 

0  =  divuj 


for  i  =  1, s.  Moreover,  we  will  have  the  following  boundary  conditions 


Uj  (?,t)  =  0,  ?  £dDi 

-^(?d)  =  o,  £edD2. 


5.2.4  Evolution  of  the  stochastic  coefficients  Y)  (x,  t]u) 

The  set  of  evolution  equations  for  the  stochastic  coefficients  will  take  the  form  of  a  SDE 


dY 

dt 


-Ypm  +  ^ Aum  -  um.Vu  -  u.Vum  -  / k  x  um,  Uj^  Ym 
(\-Ypmn  +  —  Um.Vun  +  -U„.Vum,  Uj^  (YmYn  —  ^Ym(t)Yn{t)) 


+  (-Vbr  +  (fir  (x,  t)  ,  Uj)  Zr  (■ t ;  co) 


Using  Gauss  theorem  [167]  we  have  for  every  scalar  field  a  and  every  divergent-free  vector 


field  F 


J  Va  (x)  F  (x)  dx  =  j  a  (0  F  (0  n  (?)  d? 

D  dD 


Using  this  property  for  the  velocity  field,  for  part  of  the  boundary  dD\  where  homogeneous 
Dirichlet  boundary  conditions  hold,  the  above  integral  vanishes  because  of  the  zero  velocity 
of  the  modes  on  the  boundary.  Moreover,  the  zero  boundary  pressure  associated  with  modes 
implies  that  the  above  integral  will  also  vanish  over  the  boundary  dD2.  Therefore,  we  will 
have 

(Vpm,Uj)  —  (Vpmn,Uj)  —  (V6r,  Uj)  —  0  (5.8) 


Hence,  the  stochastic  equation  describing  the  evolution  of  the  stochastic  coefficients  will  be 
take  the  form 


dt 


^Aum  -  um.Vu  -  u.Vum 


/  kx 


U-m  ,  U  i 


y 

J-  rr, 


-  (um.Vu„  +  un.Vum,  Uj)  (YmYn  -  C Ym(t)Yn(t)) 


+  {(pr  (x,  t)  ,  Uj)  Zr  (t;  uj) 


We  emphasize  that  the  last  property  (5.8)  is  crucial  since  it  allows  us  to  apply  projection 
methods  (Guermond  et  al.,  [55],  Ueckermann  &  Lermusiaux  [personal  communication])  for 
the  numerical  solution  of  the  DO  equations  for  Navier-Stokes.  In  this  numerical  framework 
the  stochastic  pressure  does  not  need  to  be  determined  completely  since  the  non-divergent 
requirement  for  the  velocity  field  is  enforced  numerically.  The  last  property  that  the  pressure 
terms  do  not  show  up  in  the  evolution  equation  for  the  stochastic  coefficients,  allows  the 
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applicability  of  projection  methods,  leading  to  significantly  smaller  computational  cost. 


5.3  The  case  of  stochastic  boundary  conditions 


We  shall  now  consider  the  problem  of  stochastic  boundary  conditions.  We  assume  that  the 
complete  stochastic  information  for  the  boundary  conditions  is  known.  More  specifically 
we  have 


u  (£,£;  to)  =  udDl  (£,f)  +  u'dDi  (£,t;  u>) ,  £  edD1 

Qu 

—  (£,£;  to)  =  h dD2  (£,t)  +  h'dD2  (£,£;  u>) ,  £  €dD-2 

where  [u 'dD  (£,i;  w) , h gD  (£,f;w)]r  is  the  zero-mean  stochastic  part  of  the  boundary  con¬ 
ditions.  We  consider  the  correlation  operator  associated  with  the  boundary  conditions 


CdDdD  (6 ,£2)  =  e<jJ 


- 1 

£ 

b 

jb 

’  U9D!  (£2 -*;w)  ' 

T" 

L  Kd2  (6^;^)  J 

.  hdD2  (£2  ,*;<*>)  _ 

We  formulate  the  eigenvalue  problem  to  determine  the  principal  directions  where  the  prob¬ 
ability  is  distributed  in  the  variance  sense 


J cdD9D  (6 ,6) 

dD 


uaDi.fc  (£2  ,t) 
hdD2,k  (£2 ,t) 


uaz?i,fe  (£i,i) 

hdD2,k  (£1  ,*) 


Using  this  information  we  expand  the  stochastic  boundary  conditions  as  follows 


u  (£,t;  u)  =  u dDl  (£,t)  +  Ek  (t;  u)  u dDl,k  (£,*) ,  £  € dD1 

du 

—  (£,i;  u)  =  h dE>2  (£,t)  +  Ek  (t;  u)  haD2)k  (£,t) ,  £  £dD2 

where  k  is  an  index  taking  values  from  1, ...,  K,  the  order  of  the  truncation,  and  the  stochas¬ 
tic  coefficients  are  given  by 

uaD!  (£,*;  W)T  udDUk  (£,*)  rf£+  y  (£,*;  w)r  hao2,fc  (£,*)  d£. 

We  will  transform  the  above  problem  to  an  equivalent  one  having  deterministic  boundary 
conditions.  In  this  case  the  effect  of  stochastic  boundary  conditions  will  come  through  the 
forcing  terms.  The  idea  is  to  handle  the  effect  of  stochastic  boundary  conditions  through 
the  partition  of  the  solution  into  a  component  u h  (x,  t;  lo)  that  will  satisfy  the  deterministic 
part  of  the  boundary  conditions,  and  a  set  of  incompressible  and  irrotational  components 
(x,  t)  that  will  satisfy  the  stochastic  part  of  the  boundary  conditions.  Specifically,  we 
write  the  solution  of  the  system  as 


u  (x,  t;  ui)  =  uh  (x,  t;  oj)  +  Ek  (t;  ui)  ubjk  (x,  t) 
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Since  the  velocity  fields  ^  (x,  t)  have  been  assumed  irrotational  and  incompressible  there 
will  be  a  set  of  scalar  potentials  (f>b,k  (x,  t)  such  that 

Ub,k  (x,  t)  =  V<t>b,k  (x,t) 

A 4>b,k  (x,  t)  =  0 

Moreover,  each  potential  function  <pb,k  (x, 1)  will  satisfy  the  following  boundary  conditions 

V06,jt  (£,t)  =  UdDltk  (€,t) )  £ 

(£,i)  =  hgD2jfc  (£,f) ,  £  £dD2 

Note  that  time  in  the  above  elliptic  equation  act  as  a  parameter;  thus  there  is  no  need  for 
initial  conditions.  With  the  above  choice  we  have  a  well  defined  problem  for  the  potentials 
<f>b  k  (x,  t)  and  additionally  our  solution  satisfies  the  stochastic  part  of  the  boundary  con¬ 
ditions.  Moreover,  we  require  u (x,  t;  uj)  to  satisfy  the  deterministic  part  of  the  boundary 
conditions  and  we  obtain  the  following  problem  for  (x,  t;  uj) 

d 

-  —  (Ek  (t;  u)  V<t>b,k  (x,  t)) 

c  edDt 
C  edD2 

and  initial  conditions 

uh  (x,t0-,u)  =  u0  (x;  u)  -  Ek  (t0;  a;)  V(f>b,k  (x,t0) ,  xeD,  wGH 

Therefore,  we  have  transformed  the  general  problem  to  one  with  deterministic  boundary 
conditions  and  stochastic  forcing.  We  emphasize  that  the  handling  of  the  stochastic  bound¬ 
ary  conditions  through  the  boundary  is  of  special  importance  for  the  case  of  systems  where 
the  initial  state  is  deterministic  and  uncertainty  is  introduced  only  through  the  boundary 
conditions  (i.e.  the  stochastic  subspace  is  initially  an  empty  set).  In  this  case  the  modes 
required  to  describe  the  current  state  of  the  system  may  be  very  few  compared  to  those 
required  to  satisfy  the  stochastic  boundary  conditions.  Using  the  above  decomposition  we 
create  a  new  set  of  modes  that  depend  exclusively  on  the  stochastic  characteristics  of  the 
boundary  conditions  and  not  on  the  system  state.  Hence,  in  this  formulation  the  stochastic 
boundary  conditions  are  satisfied  a  priory  (since  we  have  solved  for  the  potentials  (pbik  (£,  t)) 
and  we  only  need  to  solve  for  the  uncertainty  of  the  solution  in  the  interior  of  the  domain. 


0uh 

dt 


=  C  [uft  (x,  t;  u)  +  Ek  (t;  uj)  Vc^fc  (x,  t) ;  uj] 


0  =  div  Ufr 

with  deterministic  boundary  conditions 


W  (Z,t\uj)  =  udDl  (£,f) 
duh 


dn 


(£,t;w)  =  h dD2  (£,t) 


5.4  Unstable  perturbations  for  Navier-Stokes  equations 

As  we  saw  in  Chapter  4  the  application  of  adaptive  criteria  for  the  dimensionality  of  the 
stochastic  subspace  requires  the  determination  of  the  most  unstable  perturbations.  These 
perturbations  should  be  normal  to  the  current  form  of  the  stochastic  subspace  Vs .  In  Chapter 
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4  a  closed  expression  for  the  determination  of  the  most  unstable  direction  was  derived  in 
terms  of  the  Frechet  derivative  of  the  system  operator.  In  this  section  we  compute  explicitly 
the  form  of  the  unstable  mode  and  we  prove  that  this  depends  only  on  the  mean  of  the 
current  state,  while  the  dependence  through  the  DO  modes  is  indirect  through  the  fact  that 
the  unknown  mode  should  be  normal  to  every  existing  DO  mode. 

We  consider  Navier-Stokes  equations  (5.1)  and  by  computing  the  Frechet  for  the  evo¬ 
lution  operator  C  we  obtain  the  variation  towards  the  deterministic  state-space  direction 
0  (x) 

5C  [u(«,t;u/)]  =  _^dp  +  1 M  x  _  tf.vu  -  u.V0. 

ou  du  Re 

Moreover,  the  continuity  equation  will  take  the  form  (since  it  must  be  satisfied  for  all 
variations  du) 

div  0  (x)  =  0 

from  which  we  can  determine  the  variational  derivative  of  the  pressure  ^ 

dp 


A  [  j-  )  =  V-  (  -/ k  x  0  -  0.Vu  -  u.V0  )  . 


Applying  the  mean  value  operator  we  have 

5p 


A  E 


du 


=  V-  -/ k  x  0  -  0.Vu  -  u.V0 


Moreover, 

E“ 


dC  [u  (•,  t;  w)] 

du 


=  —V  E 


dp 

du 


H - A0  —  / k  x  0  —  0.Vu  —  u.V0. 

Re 


Hence,  from  Chapter  4,  Section  2  we  will  have  the  form  of  the  functional  Q  [0]  from  which 
we  will  determine  the  most  unstable  mode 


Q  [0]  =  (  E1 


dC,  [u  (•,  f;  uj)  ;  uj] 
du 


[0  (•,*)] 


,  0  (•,  0) 


dp 

du 


1 


,0>  +  — (A0,0) 


—  (0.Vu,  0)  —  (u.V0,  0)  —  (  / k  X  0,  0 


The  last  Coriolis  term  (  / k  x  0, 0  )  vanishes  identically.  Moreover,  by  using  Gauss  identity 
we  have 


V  E 


dp 

<5u 


,V)  =  -JE“ 

D 

=  f  Eu 

dD 


dp 

du 

dp 

du 


div  0dx+  /  Eu 

dD 

0.n  dS 


dp 

du 


0.n  dS 


where  the  last  equality  follows  from  the  non-divergence  property  of  0.  Note  also,  that  using 
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same  arguments  with  those  used  in  the  previous  section  we  can  show  that  /  Eu 


Sp 

(5u 


dD 


tf.n  dS  = 


0.  Thus  we  can  avoid  the  solution  of  an  inverse  (p  —  u)  problem  and  express  Q  [i?]  explicitly 
in  terms  of  the  current  mean  velocity  state  of  the  system 

Q  [$]  =  -t-  (A$,  i?)  —  (i?.Vu,  $)  —  (u.V-i?,  $} 

Re 

Hence,  the  following  expression  for  the  matrix  Qij  can  be  written 


1 


Qij  =  ^  dj)  -  (lij.Vu,^)  -  (u.'V'&i,  $j) 


Following  the  analysis  of  Section  4.2.2  we  have  an  expression  for  the  most  unstable  mode 


■&c  (x,  t)  =  aC)idi  (x,  t) 

where  acj  is  the  eigenvector  that  corresponds  to  the  maximum  eigenvalue  of  the  matrix 

Qij  T  Qji- 

We  emphasize  that  in  the  computation  of  the  matrix  Qij  used  for  the  determination 
of  the  most  unstable  mode  there  is  no  contribution  of  the  Coriolis  terms  (Coriolis  force  is 
locally  orthogonal  to  the  flow).  This  is  physically  justified  by  the  fact  that  Coriolis  force 
does  not  change  the  energy  content  of  the  system  (it  is  a  rotation  term)  and  here  the  norm 
that  we  consider  is  the  kinetic  energy.  However,  other  norms  may  be  used  to  characterize 

,he  -  unstable  perturbation.  e.g.  growth  of  enstrophy,  |/ |V  x  u|2  <ix.  In  the  latter 

D 

case,  we  will  have  a  contribution  of  the  Coriolis  force  due  to  the  spatial  variation  of  the 
Coriolis  coefficient. 


5.5  Transfer  of  energy  in  Navier-Stokes 


We  will  now  study  energy  exchange  properties  between  different  DO  modes  and  the  mean 
ffow.  As  we  saw  in  Chapter  3  the  DO  modes  remain  always  orthogonal.  Spatial  orthog¬ 
onality  of  these  fields  implies  orthogonality  of  their  spatial  Fourier,  Gabor,  and  Wavelet 
transforms  [5],  [35].  Therefore,  different  DO  modes  always  contain  different  frequency- 
phase  content  at  the  same  spatial  locations.  In  what  follows  we  will  prove  that  the  transfer 
of  energy  from  the  mean  flow  to  the  DO  modes  occurs  through  a  linear  mechanism  and  is 
triggered  by  linear  instability  of  the  mean  flow.  This  causes  transfer  of  energy  content  from 
the  mean  flow  to  the  DO  modes  in  the  form  of  variance. 

On  the  other  hand,  flow  of  energy  among  different  DO  modes  occurs  both  in  a  linear 
and  a  non-linear  way  and  is  exclusively  connected  with  the  non-Gaussian  statistics  of  the 
stochastic  coefficients.  As  we  will  see,  since  different  modes  contain  different  frequency- 
phase  content,  the  mutual  interaction  of  the  DO  modes  occurs  in  triads  and  it  is  not 
possible  to  specify  the  exact  amount  of  energy  transferred  from  one  mode  to  another  (we 
can  only  specify  the  total  energy  transferred  from  one  mode  to  two  others). 

To  illustrate  these  properties  we  consider  a  system  that  is  not  externally  forced  and 
which  has  homogeneous  Dirichlet  boundary  conditions  everywhere.  More  specifically,  we 
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consider  the  system 


(5.9) 

(5.10) 

(5.11) 


(9l_l  1 

—  =  —'S/p  +  —  Au  —  u.Vu  —  /k  x  u  =  C  [u  (x,  t ;  a;)] 
ot  Re 

0  =  divu 

u  (£,i;  uj)  =  0,  £  edD 

u  (x,  to!  w)  =  Uq  (x;  uj)  ,  x£D,  wed 


5.5.1  Energy  exchanges  between  the  DO  modes  and  the  mean  flow 


To  study  the  flow  of  energy  among  the  mean  flow  and  the  DO  modes  we  consider  the  DO 
equation  for  the  stochastic  coefficient  Y)  (f;  uj)  since  the  fields  Uj  (x,  t)  are  normalized.  We 
have,  using  Einstein  notation 


dt 


^Aum  -  um.Vu  -  u.Vum 


/  kx 


U-m  ,  U-7 


Y 

J-  rr, 


-  (um.Vu„  +  un.Vum,  Uj)  ( YmYn  -  C Yrn(t)Yn{t)) 


We  assume  that  at  the  current  time  instant  the  correlation  matrix  C Ym(t)Yn(t)  is  diagonalized 
(in  this  way  we  have  energy  on  the  diagonal  components  only)  and  we  wish  to  study  the 
transfer  of  energy  from  the  mean  flow  to  the  mode  i.  Multiplying  with  Y,  and  applying  the 
mean  value  operator  we  obtain 


[F*2]  =  (  ^Au*  ~  u*-Vu  -  u.Vuj  -  /kx  Uj,  Uj^>  Eu  [Y)2] 

2  T  Un.Vum,  Uj)  E  [EEiiYn] 

We  have 

(u.Vuj,  Uj)  =  (u,  V£j)  =  0  (5.12) 

where  Ei  =  \  (uj,Uj)  and  the  last  equation  follows  from  Gauss  identity  and  the  chosen 
boundary  conditions.  Additionally  we  have 


^/k  x  Uj,Uj^  =  0 

and  from  Gauss  theorem  and  the  chosen  boundary  conditions 


(Auj,  Uj)  =  -  (Vuj,  Vuj) . 


Finally,  we  observe  that 


(uj.Vu,  Uj) 
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where  {SQ}^  =  \  (j^  +  •  Hence,  we  will  have 


1  d_ 

2  dt 


Eu  [Y2] 


1 

Re 


(Vuj,  Vuj) 


2  (Wn-Vun  "I-  Un.Vum,  Uj)  E  \Y{Ym Yn] 


(5.13) 


We  observe  that  energy  transfer  between  the  stochastic  mode  i  and  the  mean  flow  occurs 
in  a  linear  way  although  the  terms  in  the  original  equation  which  are  responsible  for  this 
energy  transfer  are  the  nonlinear  ones  (it  is  the  quadratic  terms  in  Navier-Stokes  that  lead 
to  the  term  u^  SQUj  in  equation  (5.13)).  Hence  we  have  for  the  viscous  dissipation 


£diss,i  =  ~^eEU’  [ Yi 2]  (VUj,  VUj) 

and  the  rate  of  energy  transferred  to  or  from  the  mean  flow  to  mode  i  in  the  form  of 
stochastic  energy  (variance) 


£mean—>i  =  ~E“  [Y2]  j ufSQUjdx. 

D 

We  note  that  for  small  energy  amplitudes  Ew  [Yj2]  (so  that  terms  of  O  (T3)  can  be  omitted) 
these  two  terms  are  those  that  mainly  characterize  the  total  energy  variation  of  the  mode 
Uj,  i.e.  the  total  rate  of  energy  change  is  given  by 

^linear, i  =  £ diss,i  “r  £meon-*i 

ufSnu.dx  [Y2] 

\  D  / 

This  is  exactly  the  quantity  that  we  maximize  in  order  to  choose  the  most  unstable  mode 
for  the  adaptation  process  described  in  Chapter  4  and  in  the  previous  section.  Note  that  in 
this  case  the  added  mode  has  very  small  amplitude  so  the  hypothesis  that  terms  of  O  (T3) 
can  be  neglected  is  justified.  We  note  that  for  the  case  where  stochasticity  is  introduced 
only  through  the  initial  conditions,  uncertainties  are  always  reduced  by  the  diffusion,  but 
are  either  amplified  or  tapered  by  the  nonlinear  stretching  of  the  mean  flow. 

5.5.2  Energy  exchanges  between  the  DO  modes 

To  study  energy  exchanges  among  various  modes  we  consider  equation  (5.13)  derived  in  the 
previous  subsection.  By  inspection,  we  observe  that  the  rate  of  energy  transferred  to  mode 
i  from  all  the  DO  modes  is  given  by 

&DO— >i  =  —  2  (Um-Vun  +  U„.Vum,  Uj)  E  [Yjl'rnynJ  • 

As  it  can  be  clearly  seen  the  transfer  of  energy  among  different  DO  modes  depends  on 
the  non-Gaussian  characteristics  of  the  probability  measure  and  for  the  Gaussian  case  it 
vanishes  (for  Gaussian  variables  we  always  have  Ew  [YiYmYn]  =  0).  Note  that  the  m  =  n  =  i 


=  _  '  Re  (Vu*’  Vu*)  + 
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term  vanishes  (since  the  covariance  operator  is  diagonal)  so  we  have  two  remaining  cases. 

The  first  case  where  we  have  energy  transferred  directly  from  mode  q  to  a  different  mode 
i  (cases  m  =  q,  n  =  i,  or  m  =  i,  n  =  q,  or  n  =  m  =  q).  For  that  case,  summing  all  non-zero 
contributions,  we  have  the  rate  of  energy  transferred  directly  from  mode  q  to  mode  i 

Sq^i  =  -  (Ug.VUj  +  Ui.VUq,  Uj)  E“  [Y?Yq]  -  1  (Ug.VUq  +  Ug.VUg,  Uj)  E“  \YqYj\ 

=  -  ((uj.Vug,  Uj)  +  (ug.Vu4,  Ui))  Eu  [Y?Yq]  -  (u,.Vu„  u<)  Eu  [ Y2Yr ] 

=  -2  ^(uj.Vug,  Ui)  +  i  (uq,  V  |ui|2))  E“  [Y?Yq\  -  (uq.Vuq,  m)  E“  [Y2Yi] 

=  -  (Uj.VUg,  Ui)  Eu  [Y?Yq]  +  (u9.Vui,  u,)  E“  [ Yq2Y } 

=  -E“  [Y?Yq\  j ufSu,Uirfx+£^  [Yq2Yi\  j ujs^u 

D  D 

In  the  above,  we  have  used  the  equality  (ug.Vug,  Ui)  =  —  (ug.Vui,  ug)  which  follows  from 
direct  application  of  Gauss  identity. 

The  remaining  second  case  occurs  when  the  energy  transferred  to  mode  i  is  due  to  its 
triple  interaction  with  every  other  pair  of  DO  modes.  The  rate  of  energy  transfer  due  to 
this  triple  interaction  with  modes  p  and  q  has  the  form 

£pq—>i  —  2  (Up.VUg  T  Ug.VUp,  Ui)  E  [FiFphg] 

=  ^  ((ug.Vup,  Ui)  +  (up.Vug,  Ui))  Ew  [YiYpYql 
=  ^  ((ug.Vui,  Up)  +  (up.Vui,  Ug))  E w  [ YjYpYq ] 

=  ^  |  J u^SUiUpdx  +  J UpSU;Ugdx  j  E“  [ YiYpYq } 

\d  d  J 

=  j u^SUiu pdxE“  [YiYpYq]  . 
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5.5.3  Stochastic  energy  in  Navier-Stokes 

In  the  last  two  subsections  we  derived  expressions  characterizing  the  transform  of  energy 
to  uncertainty  (flow  of  energy  from  the  mean  to  the  modes)  but  also  the  variance  exchange 
between  the  modes.  Motivated  by  these  results  we  define  the  following  form  of  stochastic 
energy  where  energy  of  the  mean  and  variance  of  the  modes  are  considered  in  a  unified  way 

£s  =  E “  Ku,u>]  =  \EU3  [(u+TjUi,  u+liUi)] 

=  )(lM2  +  X>“  ft2]). 
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The  next  step  is  to  study  the  evolution  of  the  above  quantity.  We  have  using  the  DO 
equations 


d£i 

dt 


E“[(u,ut)}+Eu 


Yi- 


dt 


E“  [(u,  E“  [£])]  +  E“  [Yi  (C-EP  [C] ,  u*)] 
E*[(vl,E?  [C])]  +  E“  \Yi{£,Ui)] 
E“[{£,u)}  +  E“  [(A^W)] 

E“  [(^u+YiUi)} 

£“[(Au)] 


Now,  by  directly  computing  the  above  quantity  using  Gauss  identity  (and  equation  (5.9)) 
we  obtain 


d£ S 
dt 


Eu  [(£,  u)]  =  Eu 


Re 


Au,  u 


[(Vu,Vu)] 

~i~c  ((Va’ Va>  +  E“  lY?]  (Vui,  Vui» 
+  ^  [A2]  livu.il2) 


Thus,  the  stochastic  energy  for  homogeneous  Navier-Stokes  is  dissipated  due  to  viscosity  in 
full  analogy  with  the  usual  notion  of  energy  for  deterministic  Navier-Stokes.  All  the  other 
forms  of  energy  transfer  from  the  mean  flow  to  the  DO  modes  and  among  the  DO  modes 
are  internal  system  interactions. 

A  summary  of  all  the  energy  transfers  is  given  in  Figure  5-1  where  the  internal  interac¬ 
tions  among  the  DO  modes  (green  and  blue  arrows)  are  shown.  The  black  arrows  show  the 
energy  exchanges  between  the  mean  flow  and  the  modes.  Finally,  the  red  arrows  represent 
the  energy  dissipation  due  to  viscosity  acting  on  both  the  mean  flow  and  the  DO  modes. 


5.6  Application  I:  Lid  driven  cavity  flow  with  stochastic  ini¬ 
tial  conditions 

As  a  first  application  we  present  the  results  of  the  DO  field  equations  applied  to  the  nu¬ 
merical  simulation  of  a  lid-driven  cavity  flow  described  by  the  Navier-Stokes  equations  in  a 
square  domain.  The  stochasticity  is  introduced  through  the  initial  conditions.  The  physical 
configuration  (Figure  5-2)  consists  of  a  square  container  filled  with  a  fluid.  The  lid  of  the 
container  moves  at  a  given,  constant  velocity,  thereby  setting  the  fluid  in  motion.  No-slip 
conditions  are  imposed  on  all  four  segments  of  the  boundary  with  the  exception  of  the  upper 
boundary,  along  which  the  velocity  u  in  the  x— direction  is  set  equal  to  the  given  lid  velocity 
Ub  to  simulate  the  moving  lid.  The  length  of  each  side  is  L  =  1  and  the  Reynolds  number 
of  the  flow  is  taken  to  be  Re  =  1000.  For  the  stochastic  computation  the  lid  velocity  is 
taken  to  be  Ub  =  1.5  while  in  the  DO  expansion  (3.10)  we  retain  5  modes  which  is  equal  to 
the  stochastic  dimension  of  the  initial  conditions.  The  flow  fields  associated  with  the  initial 
conditions  uq  (x;  uj)  are  shown  in  Figure  5-3  in  terms  of  the  streamfunction. 
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Figure  5-1:  Energy  exchanges  between  DO  modes  and  the  mean  flow  in  stochastic,  ho¬ 
mogeneous,  Navier-Stokes  equations.  The  energy  flow  from  the  mean  to  the  modes  is 
characterized  by  the  second  order  statistics  while  variance  exchange  among  the  modes  is 
characterized  by  the  third  order  statistics. 


- ► 

u  =  ub,  v  =  0 

11  =  0 

u  =  0 

v  =  0 

u  =  v  =  0 

v  =  0 

Figure  5-2:  Driven  cavity  flow,  problem  configuration. 
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Meanv(x,y  M',(x.t0)  <h2(x.1c) 


0  0  2  0.4  0  6  0  8  1  0 


V3(X.V  «V4(X.y 


Figure  5-3:  Initial  conditions  for  the  mean  and  the  basis  of  the  stochastic  subspace  Vs  in 
terms  of  the  field  streamfunction. 
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Variance/Energy 


Figure  5-4:  Evolution  of  principal  variances  af  (t) ,  i  =  1, 5  (blue  curves)  and  mean  field 
energy  (red  curve)  for  the  flow  in  a  cavity. 


By  evolving  all  parameters  of  the  system  using  the  DO  field  equations  we  compute  the 
complete  5— dimensional  probabilistic  structure  of  the  stochasticity  inside  Vs-  In  Figure  5-4 
we  show  the  evolution  of  the  principal  variances  a f  (t)  ,  i  =  1 , . . . ,  5  which  are  the  eigenvalues 
of  the  correlation  operator  C Yi(t)Yj(t)  (blue  solid  curves).  They  provide  a  direct  measure 
of  how  the  stochastic  energy  evolves  with  time.  The  red  solid  curve  is  associated  with  the 
deterministic  kinetic  energy  of  the  mean  flow  field,  i.e.  the  quantity  (u(»,t)  ,u(»,t)) .  We 
observe  that  the  stochastic  energy  decays  almost  monotonically  after  some  initial  transient 
interactions,  while  the  energy  of  the  mean  field  slowly  grows  towards  a  steady  limit.  This 
is  an  expected  behavior  if  we  consider  the  fact  that  the  deterministic  cavity  flow  possess  a 
stable  attractor  which  is  characterized  by  a  steady  velocity  field.  Therefore,  in  the  absence 
of  external  stochastic  excitation  it  is  fully  expected  to  have  convergence  of  the  system  to 
this  deterministic,  one-dimensional  attractor.  The  mean  fields  u  (x,  t)  and  orthonormal 
basis  fields  Uj  (x,  t)  ,  i  =  1, ...,  5  are  shown  in  Figure  5-5  and  5-6  for  two  different  time 
instances  both  in  terms  of  the  streamfunction  and  vorticity.  For  the  same  time  instances 
we  present  three  out  of  the  five  two-dimensional  marginals  associated  with  the  stochastic 
processes  {Yj  (t;  cu)}*=r 

Finally,  in  Figure  5-7  we  compare  the  mean  streamfunction  computed  using  the  5-modes 
DO  method  with  the  one  obtained  by  Monte-Carlo  simulation  initialized  with  the  ESSE 
methodology  and  using  250  and  500  samples.  We  observe  that  as  we  increase  the  number 
of  samples  used  for  the  Monte  Carlo  simulation  we  obtain  better  agreement  with  the  DO 
mean  estimate. 

5.6.1  Evolution  of  a  small  stochastic  perturbation 

In  this  subsection  we  will  study  the  convergence  properties  of  the  DO  methodology  for  the 
lid  driven  cavity  flow.  The  first  feature  that  we  want  to  examine  is  sensitivity  of  the  flow 
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Figure  5-5:  Mean  field  and  basis  of  the  stochastic  subspace  Vs  in  terms  of  the  streamfunction 
and  vorticity  field;  two-dimensional  marginals  of  the  five  dimensional  joint  pdf  /  (y,  t)  at 
time  t  =  2. 
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Figure  5-6:  Mean  field  and  basis  of  the  stochastic  subspace  V$  in  terms  of  the  streamfunction 
and  vorticity  field;  two-dimensional  marginals  of  the  five  dimensional  joint  pdf  /  (y,  t)  at 
time  t  =  8. 
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DO  Mean 


Figure  5-7:  Mean  velocity  field  (streamfunction)  computed  using  Monte-Carlo  method  (250 
and  500  samples)  and  the  DO  field  equations  (s  =  5  modes)  at  t  =  1. 


in  the  addition  of  an  extra  stochastic  mode  having  very  small  variance.  In  general  the 
variance  growth  of  an  initially  low-amplitude  stochastic  perturbation  will  depend  on  the 
stability  properties  of  the  dynamical  system  (as  we  saw  in  Chapter  4).  For  the  lid-driven 
cavity  flow  there  is  a  stable  attractor  where  dynamics  tend  to  converge  after  sufficiently 
large  time.  However,  during  the  initial  phase  we  have  strong  interactions  between  the  DO 
modes  and  the  mean  flow. 

We  initiate  the  system  with  four  stochastic  modes  in  the  first  case  and  with  the  same 
four  stochastic  modes  in  the  second  case  plus  one  extra  that  has  variance  <r|  (to)  =  10~6,  i.e. 
much  smaller  than  the  stochasticity  contained  in  the  first  four  modes.  We  denote  the  first 
solution  as  u  (x,  t;  uj)  =  u  (x,  t)  +  Y \  (t;  uj)  Uj  (x,  t)  and  the  second  solution  as  v  (x,  t;  uj)  = 
v  (x,  t)  -)-  (f;  uj)  Vj  (x,  t).  After  solving  the  corresponding  set  of  DO  equations  for  the 

two  cases  we  project  the  stochastic  solution  u  (x,  t;  uj)  obtained  with  the  four  modes  to  the 
stochastic  subspace  computed  in  the  second  case  using  five  modes.  The  results  are  shown 
for  two  different  time  instants  in  Figures  5-8  and  5-9.  Specifically,  the  two  upper  plots 
represent  the  streamfunction  of  the  mean  flow  for  the  case  of  four  modes  (i/ji  (x,  t))  and 
for  the  case  of  five  modes  (ip2  (x,  t)).  The  red  curves  shown  in  the  lower  plot  represent  the 
time  series  for  the  variances  af  (t) ,  i  =  1 , . . ,  5  of  the  five-modes  solution  v(x,t;uj).  The 
blue  curves  represent  the  variance  of  the  projection  of  the  four-modes  stochastic  solution 
u  (x,  t;  uj)  to  the  stochastic  subspace  computed  using  the  five  modes  solution,  i.e.  of  (t)  = 
var  [(u  (x,  t;  uj)  ,  v,  (x,  t))]  ,  i  =  1,  ..,5. 

As  we  can  observe  the  two  sets  of  curves  start  initially  very  close  having  a  very  small 
difference  due  to  initialization  error  (different  Monte-Carlo  samples).  After  time  t  =  0.2 
the  red  curve  describing  the  variance  of  the  fifth  mode  for  the  solution  u  (x,  t;  uj)  starts  to 
grow  due  to  an  internal  instability  that  causes  transfer  of  energy  from  the  mean  flow  to 
the  stochastic  perturbation  (Figure  5-8).  However,  the  mean  flows  for  the  two  cases  still 
compare  satisfactory  since  the  difference  on  the  stochastic  part  of  the  solution  is  very  small. 

For  larger  times  t  ~  1  —  2  the  variance  of  the  fifth  stochastic  mode  (lower  red  curve) 
has  increased  more  drastically  causing  an  energy  deviation  for  the  other  variance  curves  of 
the  same  order  of  magnitude  (see  Figure  5-9  e.g.  the  higher  variance  mode:  red  and  blue 
curve).  In  this  case,  even  though  the  topology  of  the  mean  flow  remain  the  same  there  are 
some  small  differences  caused  by  the  different  energy  or  variance  distribution  among  the 
stochastic  modes. 
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Therefore,  we  may  conclude  that  the  total  error  magnitude  between  the  two  stochastic 
solutions  depends  on  the  magnitude  of  the  neglected  part  in  the  KL  expansion.  Depending 
on  the  stability  properties  of  the  fluid  flow,  the  energy  or  the  variance  of  a  small  perturbation 
may  grow.  However,  the  analytical  criteria  developed  in  Chapter  4  may  be  used  to  make 
sure  that  for  every  time  instant  the  neglected  part  of  the  stochastic  solution  is  smaller  than 
an  a  priori  chosen  tolerance.  This  guarantees  that  our  stochastic  solution  will  have  a  given 
accuracy  using  the  least  number  of  stochastic  modes  at  every  time  instant.  These  criteria 
will  be  illustrated  in  the  third  application  involving  an  unstable  double  gyre  flow  which  is 
initiated  with  deterministic  initial  conditions. 

5.6.2  Convergence  with  respect  to  the  stochastic  dimensionality 

We  will  now  study  the  convergence  properties  of  the  stochastic  solution  with  respect  to 
the  number  of  used  modes.  We  use  the  same  configuration  as  previously  but  with  only 
one  initial  stochastic  mode  having  important  variance.  Then  we  solve  the  DO  equations 
each  time  using  different  number  of  modes  (the  extra  modes  are  initiated  using  very  small 
variance  relative  to  the  variance  of  the  first  mode).  In  Figures  5-10,  5-11,  and  5-12  we 
present  the  mean  flow  for  three  time  instants  over  different  stochastic  dimensionalities, 
dim  (Vs) .  We  note  that  the  general  topological  features  of  the  fluid  flow  are  common  even 
for  the  case  of  very  low  dimensional  stochastic  subspace.  However,  for  longer  times  a  good 
degree  of  convergence  is  obtained  using  larger  number  of  modes  (see  Figure  5-12). 

In  Figures  5-13,  5-14,  and  5-15  we  present  the  convergence  properties  of  the  stochas¬ 
tic  subspace  Vg.  Specifically,  we  show  the  four  most  energetic  modes  u,  (x,  t)  and  their 
associated  variance  Eu  [F)2  ( t ;  w)]  for  different  stochastic  subspace  dimensionalities.  An  in¬ 
teresting  observation  is  that  the  robustness  of  the  modes  u,  (x,  t)  is  strongly  related  with 
the  magnitude  of  their  variance.  Therefore  the  first  mode  that  corresponds  to  the  largest 
variance  seems  to  converge  much  faster  than  the  modes  that  correspond  to  lower  magni¬ 
tude  of  variance.  The  same  observation  can  be  made  for  the  time  series  for  the  variances, 
cu)]  .  Specifically,  the  time  series  describing  modes  with  larger  variance  seem  to 
be  more  robust  with  respect  to  the  stochastic  dimensionality  of  the  solution.  These  results 
are  in  agreement  with  those  obtained  with  ESSE  (e.g.  [74],  [75],  [76]). 

Finally,  in  Figure  5-16  we  present  two  forms  of  error  with  respect  to  the  number  of  used 
modes.  The  upper  plot  shows  the  instantaneous  mean  square  error  between  the  solution 
Udim  va  (xi  ui)  that  utilizes  s  stochastic  modes  and  the  solution  U12  (x,  t;  u)  that  utilizes  12 
stochastic  modes.  Therefore  we  have  in  the  upper  plot  the  time  dependent  norm 

1 1  Udim  Vs  ~  Ui2||  t  =  EU  [(UdimK  (x,  t]  Uj)  -  Ui2  (x,  t]  u)  ,  Udim  ys  (x,  t]  u)  -  U12  (x,  t\  w))] 
and  in  the  lower  plot  the  global  error 

t  dt- 

T 

As  we  are  able  to  observe  the  instantaneous  error  becomes  maximum  during  the  time  interval 
1  <  t  <  5  where  all  the  transient  dynamics  take  place.  Subsequently  the  flow  begins  to 
reach  the  steady  state  attractor  and  the  error  decays.  In  the  lower  plot  we  observe  the 
convergence  of  the  solution  with  respect  to  the  number  of  modes.  We  note  that  after  the 
stochastic  subspace  exceeds  dim  Vs  =  9  the  error  decays  rapidly  and  remains  in  very  low 
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ui2ir  = 
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Figure  5-8:  Upper  plots:  mean  flow  for  t  =  0.7  in  terms  of  the  streamfunction  for  i)  the 
four  modes  solution  and  ii)  the  five  modes  solution.  Lower  plot:  the  red  curves  represent 
the  time  series  for  the  variances  erf  (t) ,  i  =  1, 5  of  the  five-modes  solution  v  (x,  t;  to)  while 
the  blue  curves  represent  the  projection  of  the  four  modes  solution  u  (x,  t;  u)  to  the  modes 
Vj  (x,  t) . 
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Figure  5-9:  Upper  plots:  mean  flow  for  t  =  5.98  in  terms  of  the  streamfunction  for  i)  the 
four  modes  solution  and  ii)  the  five  modes  solution.  Lower  plot:  the  red  curves  represent 
the  time  series  for  the  variances  of  (t) ,  i  =  1, 5  of  the  five-modes  solution  v  (x,  t;  to)  while 
the  blue  curves  represent  the  projection  of  the  four  modes  solution  u  (x,  t\  uj)  to  the  modes 
Vj  (x,t) . 
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dim(V)=2 
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Figure  5-10:  Mean  flow  for  various  stochastic  dimensionalities  and  for  t  =  1. 


levels;  an  indication  that  for  this  problem  the  stochastic  dimensionality  of  the  solution  has 
finite  dimension. 


5.7  Application  II:  Flow  past  a  circular  disk  with  stochastic 
initial  conditions 

Here  we  consider  the  flow  past  a  disk  immersed  in  a  channel.  The  inflow  velocity  at  the 
left  boundary  has  a  parabolic  profile  with  a  maximum  value  u  =  1.5;  the  disk  measures 
d  =  1  in  diameter  and  is  situated  at  a  distance  of  1.5  from  the  left  and  1.6  from  the  upper 
boundary.  It  is  well  known  that  for  two-dimensional  flow  past  a  circular  cylinder,  the  first 
critical  Reynolds  number  is  around  Re  ~  40,  where  the  flow  bifurcates  from  steady  state  to 
periodic  vortex  shedding  [158].  Here,  we  consider  the  case  of  Re  =  100.  A  typical  realization 
for  this  case  is  shown  in  Figure  5-17.  The  stochastic  initial  conditions  are  described  by  the 
mean  field  and  the  stochastic  subspace  basis  fields.  They  are  all  shown  in  terms  of  the 
streamfunction  in  Figure  5-18. 

The  principal  variances  of  (t)  ,  i  =  1,  ...,5  (eigenvalues  of  C Yt(t)Yj(t))  (blue  solid  curves) 
and  the  kinetic  energy  of  the  mean  flow  field  (red  solid  curve)  are  shown  in  Figure  5-19. 
In  this  case,  we  find  a  more  complex  evolution  of  the  stochastic  energy  characterized  by 
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dim(V)=2  t=2  dim(V)=3 


dim(V  )=5 


Figure  5-11:  Mean  flow  for  various  stochastic  dimensionalities  and  for  t  =  2. 
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Figure  5-12:  Mean  flow  for  various  stochastic  dimensionalities  and  for  t  =  6. 


dim(Vs)=7  dim(Vg)=6  dim(Vs)=5 


Figure  5-13:  Left  column:  mean  flow  for  various  stochastic  dimensionalities  dim  (Vs)  at 
t  =  2.  The  four  right  columns  on  the  right  show  the  four  most  energetic  modes  Uj  (x,  t) 
in  terms  of  their  streamfunction  as  well  as  their  associated  variance  Eu  [l)2  (f;  w)]  as  a 
function  of  time. 
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dim(Vs)=7  dim(Vs)=6  dim(Vs)=5 


Figure  5-14:  Left  column:  mean  flow  for  various  stochastic  dimensionalities  dim  (Vs)  at 
t  =  6.  The  four  right  columns  on  the  right  show  the  four  most  energetic  modes  Uj  (x,  t) 
in  terms  of  their  streamfunction  as  well  as  their  associated  variance  Ew  [Y)2  (t;  <*>)]  as  a 
function  of  time. 
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dim(Vs)=7  dim(Vs)=6  dim(Vs)=5 


Figure  5-15:  Left  column:  mean  flow  for  various  stochastic  dimensionalities  dim  (Vs)  at 
t  =  8.  The  four  right  columns  on  the  right  show  the  four  most  energetic  modes  Uj  (x,  t) 
in  terms  of  their  streamfunction  as  well  as  their  associated  variance  Eu  [l)2  (f;  w)]  as  a 
function  of  time. 
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Figure  5-16:  Upper  plot:  instantaneous  error  of  the  solution  with  respect  to  the  number  of 
modes  used  and  time.  Lower  plot:  time  averaged  error  with  respect  to  the  number  of  DO 
modes. 
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Figure  5-17:  A  typical  realization  of  the  flow  past  a  circular  disk. 
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Figure  5-18:  Initial  conditions  for  the  mean  and  the  basis  of  the  stochastic  subspace  Vs  in 
terms  of  the  field  streamfunction. 
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Variance/Energy 


Figure  5-19:  Evolution  of  principal  variances  af  (t) ,  i  =  1, 5  (blue  curves)  and  mean  field 
energy  (red  curve)  for  the  flow  behind  a  disk. 


oscillations  and  non-monotonic  behavior.  The  evolution  of  the  kinetic  energy  associated 
with  the  mean  field  is  also  more  complex. 

The  mean  fields  u  (x,  t)  and  orthonormal  basis  fields  u ,;(x,  f),  i  =  1, ...,  5  are  shown 
in  Figures  5-20  and  5-21  for  two  different  time  instances  in  terms  of  the  streamfunction. 
For  the  same  time  instances  we  also  present  four  out  of  the  five  two-dimensional  marginals 
associated  with  the  stochastic  processes  {Yj  (f;u;)}*=1.  As  we  can  observe,  the  basis  fields 
u,  (x,  t )  are  mainly  distorted  at  locations  close  to  the  solid  boundaries  indicating  that  the 
main  interaction  of  the  stochastic  subspace  Vs  and  the  mean  flow  is  taking  place  close  to 
these  locations  and  especially  the  circular  obstacle.  This  behavior  has  also  been  reported 
in  previous  work  based  on  generalized  PC  method  ([162]).  Larger  interactions  also  occur 
where  the  mean  vorticity  is  larger  and  where  meanders  and  eddies  form  downstream. 

Finally  in  Figure  5-22  we  compare  the  mean  streamfunction  computed  using  the  pre¬ 
sented  method  with  the  one  obtained  by  Monte-Carlo  simulation  initiated  using  ESSE 
methodology  and  500  samples. 


5.8  Application  III:  Instabilities  in  the  forced  double  gyre 
flow  in  a  basin 

The  third  application  that  we  consider  is  an  idealized  model  for  the  description  of  the 
temporal  variability  of  the  wind-driven  ocean  circulation.  Specifically,  we  consider  the 
simplest  model  of  the  double-gyre  circulation,  that  is  a  barotropic,  single-layer  QG  model 
(see  e.g.  McCalpin  and  Haidvogel  [99];  Chang  et  al.  [27]).  Simonnet  and  Dijkstra  [133] 
used  this  simple  double-gyre  model  to  study  instabilities  of  non-oscillatory  modes  that  lead 
to  the  creation  of  a  low-frequency  gyre  mode. 

The  aim  of  this  section  is  to  study  the  stochastic  response  of  this  model  for  different 
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Figure  5-20:  Mean  field  and  basis  of  the  stochastic  subspace  Vg  in  terms  of  the  streamfunc- 
tion;  two-dimensional  marginals  of  the  five  dimensional  joint  pdf  /  (y ,  t)  at  time  t  =  2. 
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Figure  5-21:  Mean  field  and  basis  of  the  stochastic  subspace  Vs  in  terms  of  the  streamfunc- 
tion;  two-dimensional  marginals  of  the  five  dimensional  joint  pdf  /  (y ,  t)  at  time  t  =  4. 
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Figure  5-22:  Mean  velocity  field  (streamfunction)  computed  using  the  DO  field  equations 
(s  =  5)  and  Monte-Carlo  method  (500  samples)  at  t  =  1. 
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forcing  parameters  and  Re  regimes.  After  giving  a  detailed  description  of  the  model  as 
well  as  an  overview  of  the  deterministic  dynamics,  we  will  examine  what  is  the  effect  of  a 
very  small  stochastic  perturbation  on  the  initial  conditions  of  the  problem.  Through  the 
developed  stochastic  framework  we  will  prove  that  in  the  unstable  regimes,  as  those  are 
predicted  by  the  deterministic  theory,  the  system  converges  to  a  stochastic  steady  state 
response  which  is  characterized  by  finite  variance  that  is  smaller  than  the  energy  of  the 
mean  flow.  For  larger  Re  this  variance  increases  causing  nonlinear  interactions  between  the 
modes  and  the  mean  flow  and  resulting  strongly  non-Gaussian,  non-stationary  responses. 
The  numerical  results  that  follow  are  based  on  a  finite-volume  framework  [148]  suitably 
adjusted  to  model  the  two-dimensional  DO  equations.  Other  numerical  frameworks  such 
as  Discontinuous  Galerkin  ([147],  [149])  may  also  be  applied  for  the  solution  of  the  DO 
equations. 


5.8.1  Model 


We  use  a  barotropic  single  layer-model  [37].  The  ocean  layer  has  a  constant  density  p  and 
mean  thickness  d  and  is  confined  to  a  square  basin  of  horizontal  dimensions  Lx  L.  The  flow 
in  the  basin  is  forced  by  a  wind  stress  (tqtx ,tqtv)  ,  where  To  is  a  characteristic  amplitude. 
Both  bottom  friction  (with  coefficient  £o)  and  lateral  (with  coefficient  Ah)  are  considered. 
Using  characteristic  horizontal  and  vertical  scales  L  and  d,  a  horizontal  velocity  scale  U,  a 
wind  stress  tq,  and  a  timescale  L/U,  the  nondimensional  equations  become 
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where  Re  is  the  flow  Reynolds  number,  /  =  fo+PoV  is  the  Coriolis  coefficient,  a  the  strength 
of  the  wind  stress,  and  p  the  bottom  friction  parameter.  Their  expression  are  given  as 

D  UL  toL  a  PoL2  Le0 

Re= a= w1'  p=—' 


In  the  results  shown  below,  we  follow  Simonnet  and  Dijkstra  [133]  and  we  set  both  p 

f2L2 

and  F  =  °^d  to  zero.  The  Reynolds  number  Re  is  used  as  the  control  parameter.  The 
reference  values  of  the  other  parameters  are  indicated  in  Table  5.1  (in  dimensional  form)  and 
Table  5.2.  The  values  of  the  dimensionless  inertial  and  viscous  boundary  layer  thickness, 
5j  =  L~l  \JU I fio  and  5m  =  A-1  (Ah/P o)1^3  are  also  shown. 

The  flow  in  the  basin  is  forced  by  an  idealized  zonal  wind  stress  that  is  constant  in  time, 
given  by 

1 

Tx  = - COS  27 TV. 

x  2n  y 


Free  slip  boundary  conditions  are  imposed  on  the  northern  and  southern  walls  (y  =  0, 1) 
and  no- slip  boundary  conditions  on  the  eastern  and  western  walls  (x  =  0, 1) .  In  all  the 
results  that  follow  the  system  is  initiated  with  zero  mean  flow  and  zero  stochastic  part. 
After  a  small  time  interval  (1CD2  non-dimensional  time)  a  stochastic  perturbation  is  added 
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Table  5.1:  Reference  values  of  parameters  in  the  barotropic  QG  model  (dimensional). 


Parameter 

Value 

Parameter 

Value 

U 

7.1  x  10~3m..s_1 

L 

1.0  x  106m 

L/U 

4.46yr 

d 

2500m 

TO 

1.26  x  10_1Pa 

Po 

7.1  x  10“12  (m.s)-1 

P 

10  3Kg.m~3 

fo 

5.0  x  10-5s-1 

Table  5.2:  Reference  values  of  parameters  in  the  barotropic  QG  model  (non-dimensional). 


Parameter 

Value 

Parameter 

Value 

a 

1000 

P 

103 

F 

0 

P 

0 

61 

0.032 

5m 

0.02  -  0.04 

with  Gaussian  statistics  and  variance  equal  to  10“6  of  the  mean  energy.  The  shape  of  the 
perturbation  is  chosen  according  to  the  stability  analysis  arguments  presented  in  Section 
5.4.  Subsequently  we  use  the  adaptive  framework  developed  in  Chapter  4  to  add  extra 
stochastic  modes  if  this  is  necessary. 


5.8.2  An  overview  of  deterministic  analysis 

In  this  section  we  give  a  summary  of  the  results  presented  in  Simmonet  and  Dijkstra  [133] 
that  follow  from  deterministic  stability  analysis  of  the  system.  More  specifically,  it  is  shown 
that  the  considered  QG  model  has  a  globally  stable  attractor  for  sufficiently  low  Re  numbers. 
This  can  be  seen  in  the  bifurcation  diagram  in  Figure  5-23  (published  in  [133])  where  the 
vertical  axis  corresponds  to  the  non-dimensional  intensity  of  the  subtropical  gyre,  that  is 
V’ subtropical  =  max^x)  ip  (with  ip  being  the  flow  streamfunction)  and  the  horizontal  axis 
corresponds  to  the  Reynolds  number. 

As  the  Reynolds  number  increases  and  becomes  larger  than  the  critical  value  Re  =  29.7 
then  a  spatially  symmetric  mode  (symmetry  is  meant  in  the  sense  of  vorticity  field),  the 
P-rnode  (shown  in  Figure  5-24  top-left)  becomes  unstable  causing  the  existence  of  multiple 
steady  states.  In  Figure  5-24  (published  in  Simmonet  and  Dijkstra  [133])  the  eigenvalue 
corresponding  to  the  P-rnode  is  presented.  As  we  are  able  to  observe  for  Reynolds  number 
between  Re  =  29.7  and  Re  =  39.3  the  symmetric  mode  has  an  eigenvalue  with  a  positive  real 
part  that  leads  to  instabilities.  However,  even  in  this  unstable  regime  the  system  converges 
to  an  attractor  which,  in  contrary  to  the  low  Re  regime,  is  stochastic  (the  authors  describe 
it  as  a  multiple  steady  state  regime,  [133]). 

For  higher  Reynolds  number  the  P-rnode  becomes  stable  with  a  negative  growth-factor. 
This  is  the  case  for  higher  Reynolds  numbers  up  to  Re  =  71.5  where  the  first  Hopf  bifurcation 
takes  place  and  oscillations  begin  to  occur.  For  even  higher  Reynolds  numbers  two  more 
Hopf  bifurcations  take  place  as  it  is  shown  in  5-23  (points  Hi,  H gyre,  H3).  In  the  sections 
that  follow  we  will  give  an  exact  probabilistic  characterization  of  these  regimes  using  the 
developed  stochastic  framework. 
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Figure  5-23:  Bifurcation  diagram  for  the  QG  model  with  values  of  the  parameters  as  in 
Tables  5.1  and  5.2  (from  Simonnet  and  Dijkstra  [133]). 
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Figure  5-24:  Spectral  behavior  of  the  eigenmodes  involved  into  the  various  bifurcations  of 
the  anitsymmetric  branch  for  the  barotropic  QG  model  (from  Simonnet  and  Dijkstra  [133]). 
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Figure  5-25:  Stochastic  responce  of  the  QG  model  for  Re  =  25.  The  left  top  plot  shows 
the  mean  flow  in  terms  of  the  vorticity  (colormap)  and  the  streamlines  (black  solid  curves). 
The  right-top  plot  indicates  the  energy  of  the  mean  flow  (red  curve)  and  the  variance  of 
the  stochastic  mode.  The  bottom  plot  indicates  the  first  DO  mode  together  with  the  pdf 
for  the  stochastic  coefficient. 


5.8.3  Bifurcation  analysis  of  the  stochastic  response 

We  begin  our  probabilistic  analysis  in  the  stable  regime,  i.e.  Re  <  29.7.  In  Figure  5-25 
we  present  the  stochastic  response  of  the  system.  The  left-top  plot  shows  the  mean  flow  in 
terms  of  the  vorticity  (colormap)  and  the  streamlines  (black  solid  curves).  The  right-top  plot 
indicates  the  energy  of  the  mean  flow  (black  curve)  and  the  variance  of  the  stochastic  mode 
(blue  curve).  The  bottom  plot  indicates  the  first  DO  mode  together  with  the  probability 
density  function  for  the  stochastic  coefficient.  In  this  case  as  we  are  able  to  observe  the 
system  state  is  stable  and  the  system  rapidly  converges  to  a  deterministic  attractor  since 
the  variance  of  the  stochastic  perturbation  decays  monotonically.  Therefore,  in  this  regime 
the  system  behaves  deterministically. 

By  numerically  solving  the  DO  field  equations  over  various  Re  numbers  we  found  that  the 
deterministic  regime  is  extended  up  to  Re  ~  30.2.  A  possible  explanation  for  the  numerical 
difference  with  the  results  by  Simmonet  and  Dijkstra  [133]  is  the  different  numerical  schemes 
used  but  also  the  fact  that  very  small  linear  growth  rate  may  not  be  able  to  cause  finite 
size  stochastic  perturbations. 

As  we  increase  the  Reynolds  number  we  observe  in  Figure  5-26  that  the  variance  of 
the  stochastic  perturbation  converges,  in  an  oscillatory  manner  (see  Figure  5-27),  to  a 
steady  state.  In  this  case  we  have  one  way  interaction  between  the  mean  flow  and  the 


101 


Mean  flow 


Time  =  1 .40 


Re  =  35 


Figure  5-26:  Stochastic  responce  of  the  QG  system  for  Re  =  35  after  the  system  has  reached 
steady  state  statistics. 


stochastic  perturbation,  since  the  steady  state  variance  of  the  stochastic  mode  is  pumping 
energy  from  the  mean  flow  (since  energy  is  continuously  dissipated  in  the  DO  mode  due  to 
viscosity)  while  on  the  same  time  the  mean  flow  preserves  its  antisymmetric  character,  i.e. 
it  is  not  influenced  by  the  symmetric  perturbation.  The  statistics  seem  to  preserve  their 
original  character,  i.e.  they  have  normal  distribution.  The  spatial  shape  of  the  stochastic 
perturbation  is  strongly  related  with  the  linearly  unstable  mode  computed  in  [133]  (see 
Figure  5-24),  i.e.  the  tripolar  form  of  the  perturbation  presented  in  Simmonet  and  Dijkstra 
[133].  In  this  way  we  have  obtained  the  complete  statistical  information  for  this  family  of 
multiple  equilibria,  reported  in  Simmonet  and  Dijkstra  [133].  Also,  we  emphasize  that  the 
steady  state  statistics  do  not  depend  on  the  initial  magnitude  of  the  perturbation.  The 
magnitude  of  the  variance  for  the  steady  state  regime  increases  monotonically  with  the 
Reynolds  number.  At  Re  ~  38  this  growth  is  interrupted  by  the  mutual  interaction  of  the 
stochastic  perturbation  with  the  mean  flow.  More  specifically,  in  Figure  5-28  we  present  the 
initial  regime  of  the  dynamics  for  Re  =  38.  At  time  t  =  3.46  the  existing  stochastic  mode 
exceeds  the  critical  limit  for  variance  causing  the  addition  of  a  new  mode  (as  it  is  described 


102 


t  =  0.66 


t  =  0.67 


t  =  0.70 


t  =  0.71 


Figure  5-27:  Oscillation  of  the  DO  mode  during  convergence  to  the  steady  state  attractor 
for  Re  =  35. 
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Mean  flow  Time  =  3.50  Re  =  38 


Figure  5-28:  Addition  of  an  extra  mode  at  the  time  instant  where  the  existing  mode  exceeds 
the  predefined  variance  (Flow  Re  =  38). 


in  Chapter  4).  This  new  mode,  however,  tends  to  decay,  in  an  oscillatory  manner  as  it  is 
illustrated  in  Figure  5-29. 

The  P-rnode  continuous  to  grow  up  the  point  where  its  variance  becomes  comparable 
with  the  energy  of  the  mean  flow.  At  this  point  we  have  a  two-way  interaction  between 
the  stochastic  perturbation  and  the  mean  flow  causing  decay  of  the  mean  flow  energy. 
Interestingly  both  the  mean  and  the  stochastic  perturbation  preserve  their  symmetric  char¬ 
acteristics  although  both  are  deformed  relative  to  the  regime  where  their  energies  were  not 
comparable.  Subsequently,  the  system  reaches  a  steady  state  that  is  time  independent.  This 
behavior  is  consistent  with  the  description  given  in  Simmonet  and  Dijkstra  [133].  Here,  we 
have  also  derived  an  exact  description  of  the  flow  characteristics  for  the  stochastic  steady 
state  regime. 

At  even  higher  Re  number  (Re  =  40)  the  second  mode  becomes  unstable  causing  its 
variance  to  grow  similarly  with  the  first  mode.  An  interesting  feature  is  the  symmetry 
properties  of  the  DO  modes  (Figure  5-32).  We  observe  that  the  first  one  is  spatially  an¬ 
tisymmetric  while  its  pdf  retains  its  symmetric  structure.  The  second  mode  however  has 
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Mean  flow  Time  =  6.38  Re  =  38 


Time 


Figure  5-29:  Mode  removal  due  to  very  low  variance  (Flow  Re  =  38). 
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Mean  flow 


Time  =  12.10 


Re  =  38 


Figure  5-30:  The  variance  of  the  stochastic  perturbation  becomes  comparable  with  the 
energy  of  the  mean  flow.  At  this  point  mutual  interactions  between  the  mean  flow  and  the 
perturbation  begin  to  occur. 
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Mean  flow 


Time  =  1 5.23 


Re  =  38 


Figure  5-31:  Steady  state  regime  for  Re  =  38.  Note  that  the  perturbation  retains  its 
symmetric  character  even  though  the  variance  of  the  mode  is  comparable  with  energy  of 
the  mean  flow. 
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Mean  Flow 


Variance /Energy  T  =  5  Re  =  40 


Figure  5-32:  Stochastic  response  for  Re  =  40.  Note  that  the  first  mode  has  symmetric 
spatial  properties  which  are  accompanied  by  symmetric  pdf  while  the  second  mode  has 
antisymmetric  spatial  properties  with  non-symmetric  pdf. 


the  same  type  of  symmetry  with  the  mean  flow  (antisymmetric)  while  the  corresponding 
stochastic  coefficient  has  lost  its  initially  symmetric  structure  (initially,  when  we  add  a  new 
mode  its,  density  is  assumed  to  be  Gaussian  -  see  Chapter  4). 

For  higher  Re  number  (Re  =  55)  we  obtain  again  stochastic  instabilities.  In  this  case, 
however,  we  observe  that  the  mean  flow  has  different  characteristics  with  the  vortices  being 
completely  detached  from  the  left  boundary.  The  response  is  initially  Gaussian  (Figure 
5-33)  but  as  time  evolves,  the  statistics  become  more  complicated  with  the  first  mode  being 
described  by  a  bimodal  distribution  (Figure  5-34).  Therefore,  in  this  case  the  ffow  has  two 
nrost-probable  stages  around  which  the  probability  measure  is  distributed.  Those  stages 
are  described  completely  through  this  approach.  We  emphasize  that  numerical  simulations 
(which  we  do  not  present  here)  confirm  that  this  picture  is  robust  in  terms  of  the  initial 
distribution  magnitude.  More  specifically,  the  shape  of  the  modes  as  well  as  the  shape 
of  the  distribution  remains  qualitatively  the  same  independent  of  the  initial  magnitude 
of  the  perturbations  confirming  that  the  observed  dynamics  are  caused  by  the  nonlinear 
interactions  with  the  mean  flow  and  they  are  not  the  result  of  the  growth  of  the  perturbation 
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Figure  5-33:  Initial  stage  of  motion  for  Re  =  55. 


on  the  initial  conditions. 

As  we  increase  the  Reynolds  number  (exceeding  Re  =  65),  the  flow  loses  its  stochastic 
features  converging  again  to  a  deterministic  attractor  that  is  different  from  the  one  observed 
in  the  lower  deterministic  regime.  The  results  are  shown  for  Re  =  65  in  Figure  5-35.  More 
specifically,  in  this  regime  the  two  gyres  fill  completely  the  domain  while  the  stochastic 
perturbation  tends  to  zero  after  an  initial  transient  regime.  The  above  behavior  is  consistent 
with  the  deterministic  analysis  that  predicts  stability  for  this  regime.  Note,  that  as  we 
increase  the  Reynolds  number  the  convergence  of  the  stochastic  mode  to  zero  occurs  more 
rapidly  (before  a  critical  Reynolds  limit  after  which  instabilities  occur)  as  it  can  be  seen  in 
Figure  5-36  and  5-37. 

For  higher  Reynolds  number  we  enter  the  regime  of  unsteady  behavior  in  the  long  term 
dynamics.  The  deterministic  analysis  predicts  the  occurrence  of  three  Hopf  bifurcations 
leading  to  periodic  responses.  Using  the  stochastic  framework  we  found  that  for  sufficiently 
small  Re  (Re  <  100)  we  enter  initially  a  stochastic  regime  that  allows  variance  to  increase 
temporarily.  This  is  followed  by  a  convergence  to  a  deterministic  attractor.  However  for 
larger  Re  (see  Figure  5-38  for  Re  =  200)  this  deterministic  behavior  is  interrupted  by  a 
sudden  growth  of  the  stochastic  perturbation  with  energy  pumped  directly  from  the  mean 
flow.  This  is  followed  by  the  addition  of  more  DO  modes  according  to  the  criteria  developed 
in  the  previous  Chapter.  As  we  observe  in  this  case  the  responses  become  strongly  non- 
Gaussian  illustrating  very  clearly  the  multiple  equilibria  that  the  system  may  reach. 

The  results  presented  in  this  section  are  summarized  in  the  diagram  of  Figure  5-39.  As 
we  are  able  to  observe,  in  agreement  with  the  deterministic  analysis,  the  system  is  charac¬ 
terized  by  two  deterministic  regimes.  Between  these  two  regimes  we  have  the  existence  of  a 
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Figure  5-34:  As  time  evolves  more  modes  have  to  be  added  in  order  to  achieve  the  given 
tolerance  (in  terms  of  variance). 
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Mean  Flow 


Variance  /  Energy  T  =  6  Re  =  65 


Figure  5-35:  Convergence  to  a  deterministic  attractor  after  a  transient  stochastic  regime 
for  Re  =  65. 


linearly  unstable  mode  leading  to  finite-amplitude,  time- independent,  stochastic  perturba¬ 
tions  for  which  we  obtain  an  exact  probabilistic  description.  For  higher  Reynolds  number 
we  still  have  the  occurrence  of  the  deterministic  attractor.  However,  when  the  energy  of 
the  flow  becomes  sufficiently  large  then  we  have  transfer  of  energy  from  the  mean  flow  to 
the  first  DO  mode  enhancing  its  variance.  This  leads  to  the  addition  of  more  DO  modes 
and  to  non-Gaussian  transient  responses. 

In  conclusion,  we  have  illustrated  how  the  DO  field  equations  provide  a  natural  frame¬ 
work  for  the  partition  of  the  problem  into  different  dynamical  components  each  one  con¬ 
taining  independent  information.  This  is  done  without  assuming  anything  on  the  nature  of 
the  response  or  its  energy  level.  Moreover,  the  derived  results  are  qualitatively  consistent 
with  the  results  predicted  by  linearization  of  the  system  in  the  deterministic  framework. 
The  modes  extracted  may  be  used  for  the  setup  of  further  reduced  order  models  that  take 
into  account  more  efficiently  the  mutual  interactions  of  modes  and  the  finite-amplitude  of 
the  response.  In  the  next  section  we  will  examine  the  qualitative  changes  occurring  in  the 
dynamics  for  larger  Reynolds  number. 

5.8.4  Stochastic  response  for  larger  Reynolds  number 

We  will  now  present  results  for  the  evolution  of  the  flow  with  a  much  larger  value  of  the 
Reynolds  number.  Specifically  we  consider  the  case  of  Re  =  10000.  As  we  observe  in  Figure 
5-40  in  the  initial  regime  of  the  flow  we  have  the  formation  of  two  antisymmetric  small 
vortices.  The  DO  mode  follow  these  vortices,  i.e.  it  localizes  around  them  while  its  statistics 
remain  Gaussian  during  this  initial  phase.  As  the  energy  of  the  mean  flow  increases  so  does 
the  variance  of  the  first  DO  mode  causing  the  addition  of  new  modes  shown  in  Figure  5-41. 
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Figure  5-36:  Response  for  Re  =  85.  In  this  case  the  convergence  to  the  deterministic 
attractor  occurs  earlier  with  the  initial  stochastic  regime  being  much  sorter  in  time. 
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Mean  Flow 


Variance  /Energy  T  =  8  Re  =  100 


Figure  5-37:  Convergence  to  a  deterministic  attractor  for  Re  =100. 


Similarly  with  the  first  mode,  the  two  new  modes  rapidly  localize  around  the  boundary  of 
the  vortices.  Moreover,  comparing  with  the  lower  Reynolds  number  results  we  see  that  in 
this  case  the  DO  modes  present  smaller  spatial  scales  and  sharper  gradients.  In  the  same 
time  the  variance  of  the  modes  continues  to  grow  while  the  statistics  remain  Gaussian  and 
their  symmetry  is  preserved. 

This  picture  of  Gaussian  statistics  changes  very  suddenly  with  a  nonlinear  instability 
that  causes  the  pdf  of  the  third  mode  to  lose  its  symmetry.  This  loss  of  symmetry  is 
accompanied  by  change  of  the  spatial  symmetric  properties  of  the  corresponding  mode. 
Note  that  results  (not  shown  here)  indicate  that  this  loss  of  symmetry  is  connected  with 
energy  transfer  between  in  the  modes  while  the  instabilities  presented  so  far  where  connected 
with  energy  transfer  from  the  mean  flow  to  the  modes.  Specifically  we  observe  that  the 
mode  loses  its  antisymmetric  properties  and  becomes  antisymmetric  as  the  mean  flow  is. 
This  non  Gaussian  behavior  passes  to  the  other  modes  very  rapidly  causing  the  system  to 
reach  a  multi-equilibrium  regime  described  very  efficiently  by  the  joint  pdf  function  (whose 
marginals  are  shown  in  Figure  5-43). 
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Figure  5-38:  Stochastic  attractor  after  a  temporal  convergence  to  deterministic  dynamics 
(Re  =  200). 
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Figure  5-39:  Summary  of  the  stochastic  analysis  for  the  double  gyre  flow  over  various 
Reynolds  numbers. 
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Figure  5-40:  Initial  regime  of  the  stochastic  response  for  Re  =  104. 


Figure  5-41:  The  modes  added  are  localized  around  the  boundary  of  the  formed  gyres  of 
the  mean  flow. 
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Figure  5-42:  After  the  mean  flow  energy  exceeds  a  certain  limit  an  instability  breaks  the 
symmetry  of  the  third  mode  as  it  is  shown  in  the  corresponding  pdf  plot. 
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Figure  5-43:  The  instabilitiy  shown 
non-Gaussian  statistics  shown  here. 


in  the  previous  figure  is  the  starting  point  for  the 
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Chapter  6 

Finite-size  particles  in  stochastic 
flows 


Abstract 

In  this  chapter  we  shall  study  the  motion  of  finite-size  particles  in  flows  with  uncertainty. 
Specifically,  we  will  examine  the  coupled  effects  due  to  inertia  and  flow  stochasticity.  In  the 
first  part  of  the  chapter  we  will  summarize  theoretical  results  for  particles  in  deterministic 
flows.  Subsequently  we  will  present  results  for  the  stochastic  case.  Specifically,  we  will  prove 
that  the  velocity  of  finite-size  particles  is  governed  by  a  stochastic  slow  manifold,  a  ‘layer’ 
of  probability  around  the  deterministic  slow  manifold  derived  previously  for  deterministic 
flows.  Based  on  the  stochastic  reduction  on  this  manifold  we  will  derive  a  stochastic  inertial 
equation  that  governs  the  motion  of  particles  and  which  includes  new  terms  expressing 
the  coupled  effect  of  particles  inertia  and  stochasticity.  In  the  second  part  of  the  chapter 
we  will  first  illustrate  numerically  the  convergence  of  the  particles  stochastic  velocity  to 
the  stochastic  slow  manifold.  We  will  validate  the  derived  inertial  equation  for  a  specific 
example  and  we  will  analyze  the  coupled  effects  of  particles  inertia  and  flow  stochasticity 
on  the  preferential  concentration  of  particles. 


6.1  Introduction 

Dust,  impurities,  droplets,  air  bubbles,  and  other-finite  size  particles  transported  by  in¬ 
compressible  flows  are  commonly  encountered  in  many  natural  phenomena  and  industrial 
processes.  Applications  showing  the  importance  of  the  phenomenon  are  pollutants  transport 
in  the  ocean  and  atmosphere  [146],  [40],  [62],  rain  initiation  [114],  [41],  [131],  coexistence 
between  several  species  of  plankton  in  the  hydrosphere  [119],  [84],  or  planet  formation  by 
dust  accretion  in  the  solar  system  [155],  [36].  In  all  of  the  above  cases  the  flow  velocity  field 
may  be  characterized  by  uncertainties  or  stochasticity  either  because  the  flow  dynamics 
is  not  fully  resolved  or  because  initial,  boundary  or  parametric  uncertainties  are  signifi¬ 
cant.  Therefore,  an  important  question  is  the  analytical  quantification  of  the  effect  of  flow 
stochasticity  on  finite-size  particle  dynamics. 

Several  studies  have  been  devoted  for  the  above  problem.  In  Maxey  [97]  the  gravitational 
settling  of  aerosol  particles  in  homogeneous  and  stationary  random  flow  fields  is  studied. 
Using  numerical  simulations  of  Gaussian  random  fields  it  is  shown  that  the  coupled  effect  of 
particle  inertia  and  flow  stochasticity  produces  an  increased  settling  velocity.  These  results 
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are  then  considered  in  terms  of  various  asymptotic  limits  of  either  rapid  settling  or  weak 
particle  inertia. 

Vasiliev  and  Neishtadt  [151]  consider  the  problem  of  finite-size  particle  transport  in 
steady  flows  in  the  presence  of  small  noise.  It  is  shown  that  for  the  case  of  cell  flow  this 
effect  is  important  at  high  viscosity  and  leads  to  a  transition  from  bounded  motion  of  the 
particles  to  diffusion-type  chaotic  motion. 

Reynolds  [117]  derived  for  one  dimensional  flows,  Lagrangian  stochastic  models  for  the 
prediction  of  fluid  velocities  along  heavy-particle  trajectories,  by  assuming  the  well-mixed 
condition.  This  approach  ensures  consistency  with  the  Eulerian  fluid  velocity  statistics. 
However,  for  higher  dimensional  flows  additional  assumptions  are  required  for  the  unique 
definition  of  a  Lagrangian  stochastic  model  using  this  approach.  The  derived  model  is 
applied  to  simulate  the  trajectories  of  heavy  particles  in  a  vertical  turbulent  pipe  flow. 

In  Pavliotis  et  al.  [Ill]  the  problem  of  inertial  particles  in  a  random  flow  field  with  speci¬ 
fied  structure  is  considered.  Specifically,  the  authors  study  the  case  of  a  time-dependent  flow 
with  stationary  spatial  structure  and  with  random  time  dependence  defined  by  a  stationary 
Ornestein-Uhlenbeck  process.  Using  homogenization  theory  they  prove  that  under  appro¬ 
priate  assumptions  the  large-scale,  long-time  behavior  of  the  inertial  particles  is  governed 
by  an  effective  diffusion  equation  for  the  position  variable  alone. 

Klyatskin  and  Elperin  [64]  and  Klyatskin  [65]  study  the  problem  of  diffusion  of  a  low- 
inertia  particle  in  the  field  of  a  random  force  that  is  spatially  homogeneous.  In  this  case 
the  authors  prove  that  the  problem  admits  an  analytic  solution  which  predicts  that  the 
particle  velocity  will  be  a  Gaussian  stochastic  process  with  known  covariance  function.  Bee 
et  al.  [12],  [13]  study  the  dynamics  of  very  heavy  particles  suspended  in  incompressible 
flows  with  d— correlated- in-time  Gaussian  statistics.  Under  these  assumptions  they  derive  a 
model  which  is  used  to  single  out  the  mechanisms  leading  to  the  preferential  concentration 
of  particles. 

In  what  follows  we  will  use  recent  results  from  stochastic  singular  perturbation  theory 
[18]  in  combination  with  a  Karhunen  Loeve  representation  of  the  random  flow,  in  order 
to  derive  a  reduced  order  inertial  equation  that  will  describe  the  stochastic  dynamics  of 
inertial  particles  in  arbitrary  random  flows.  As  it  has  been  observed  in  the  literature  the 
random  part  of  the  fluid  flow  changes  both  the  mean  dynamics  of  the  finite-size  particles 
(an  effect  that  is  usually  expressed  through  clustering)  but  also  their  diffusive  dynamics 
(usually  observed  through  reduced  decorellation  time  -  also  known  as  ‘crossing  trajectories’ 
effect  [164],  [33]).  Here,  our  primary  aim  is  the  study  of  the  stochasticity  of  the  flow  on  the 
mean  dynamics  of  the  particles.  Specifically,  through  the  reduced  order  stochastic  dynamics 
we  will  study  the  result  of  a  zero  mean  stochastic  perturbation  on  the  clustering  properties 
of  inertial  particles.  We  will  also  validate  and  illustrate  our  theoretical  findings  through  the 
stochastic  double  gyre  flow  presented  in  Chapter  5. 

6.2  Summary  of  results  for  finite-size  particles  in  determin¬ 
istic  flows 

In  this  section  we  will  summarize  some  recent  results  on  the  dynamics  of  finite-size  particles 
in  deterministic  flows.  These  results  will  be  essential  for  analyzing  the  motion  of  finite- size 
particles  in  stochastic  flows.  All  the  material,  as  well  as  proofs  and  applications  in  various 
settings  are  included  in  the  following  publications:  Sapsis  and  Haller,  2008a, b,  2009,  2010a, 
b  [128],  [122],  [123],  [124],  [125],  and  Haller  and  Sapsis  2008,  2010  [56],  [57]. 
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6.2.1  Reduced  order  dynamics 


Let  u(x,  t)  denote  the  velocity  field  of  a  two-  or  three-dimensional  fluid  flow  of  density 
Pf,  with  x  referring  to  spatial  locations  and  t  denoting  time.  The  fluid  fills  a  compact 
(possibly  time- varying)  spatial  region  D  with  boundary  dD ;  we  assume  that  D  is  a  uniformly 
bounded  smooth  manifold  for  all  times.  We  also  assume  u(x,  t)  to  be  r  times  continuously 
differentiable  in  its  arguments  for  some  integer  r  >  1.  We  denote  the  material  derivative  of 
u  by 

Du 

—  =  ut  +  (Vu)  u. 

Let  x(t)  denote  the  path  of  a  finite-size  particle  of  density  pp  immersed  in  the  fluid.  If 
the  particle  is  spherical,  its  velocity  v(t)  =  x(f)  satisfies  the  equation  of  motion  (cf.  Maxey 
and  Riley  [98]  and  Babiano  et  al.  [8]) 
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Here  pp  and  pf  denote  the  particle  and  fluid  densities,  respectively,  a  is  the  radius  of  the 
particle,  g  is  the  constant  vector  of  gravity,  and  u  is  the  kinematic  viscosity  of  the  fluid. 
The  individual  force  terms  listed  in  separate  lines  on  the  right-hand  side  of  (6.1)  have 
the  following  physical  meaning:  (1)  force  exerted  on  the  particle  by  the  undisturbed  flow 
(2)  buoyancy  force  (3)  Stokes  drag  (4)  added  mass  term  resulting  from  part  of  the  fluid 
moving  with  the  particle  (5)  B asset- Boussinesq  memory  term.  The  terms  involving  a2Au 
are  usually  referred  to  as  the  Fauxen  corrections. 

For  simplicity,  we  assume  that  the  particle  is  very  small  (a  <C  1),  in  which  case  the 
Fauxen  corrections  are  negligible.  We  note  that  the  coefficient  of  the  Basset  Boussinesq 
memory  term  is  equal  to  the  coefficient  of  the  Stokes  drag  term  times  a/y/Tiu.  Therefore, 
assuming  that  a/y/v  is  also  very  small,  we  neglect  the  last  term  in  (6.1),  following  com¬ 
mon  practice  in  the  related  literature  (Michaelides  [100]).  We  finally  rescale  space,  time, 
and  velocity  by  a  characteristic  length  scale  L,  characteristic  time  scale  T  =  L/U  and 
characteristic  velocity  U ,  respectively,  to  obtain  the  simplified  equations  of  motion 


3 R  Du 

T~Di 


~P  (v  —  u)  + 


(6.2) 


with 


R  = 


2Pf 

Pf  +  2  pp 


P  = 


R 

St’ 


St  = 


Re, 


and  with  t,  v,  u  and  g  now  denoting  nondimensional  variables.  Variants  of  equation  (6.2) 
have  been  studied  by  Babiano,  Cartwright,  Piro  and  Provenzale  [8],  Benczik,  Toroczkai  and 
Tel  [15],  and  Vilela,  de  Moura  and  Grebogi  [152], 
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In  equation  (6.2),  St  denotes  the  particle  Stokes  number  and  Re  =  UL/v  is  the  Reynolds 
number.  The  density  ratio  R  distinguishes  neutrally  buoyant  particles  ( R  =  2/3)  from 
aerosols  (0  <  R  <  2/3)  and  bubbles  (2/3  <  R  <  2).  In  the  limit  of  infinitely  heavy 
particles  (R  =  0),  equations  (6.2)  become  the  Maxey-Riley  equations  derived  originally  in 
[98].  The  3R/2  coefficient  represents  the  added  mass  effect:  an  inertial  particle  brings  into 
motion  a  certain  amount  of  fluid  that  is  proportional  to  half  of  its  mass.  For  neutrally 
buoyant  particles,  the  equation  of  motion  is  simply  (v  —  u)  =  —  fx(v  —  u),  i.e.,  the 
relative  acceleration  of  the  particle  is  equal  to  the  Stokes  drag  acting  on  the  particle. 

Rubin,  Jones  and  Maxey  [121]  studied  (6.2)  with  R  =  0  in  the  special  case  when  u 
describes  a  two-dimensional  cellular  steady  flow  model.  They  used  a  geometric  singular 
perturbation  approach  developed  by  Fenichel  [43]  to  understand  particle  settling  in  the  flow. 
The  same  technique  was  employed  by  Burns  et  al.  [24]  in  the  study  of  particle  focusing  in 
the  wake  of  a  two-dimensional  bluff  body  flow,  which  is  steady  in  a  frame  co-moving  with  the 
von  Karrnan  vortex  street.  Recently,  Mograbi  and  Bar-Ziv  [101]  discussed  this  approach  for 
general  steady  velocity  fields  and  made  observations  about  possible  asymptotic  behaviors 
in  two  dimensions. 

Here  we  construct  an  attracting  slow  manifold  that  governs  the  asymptotic  behavior 
of  particles  in  system  (6.2).  We  also  obtain  an  explicit  dissipative  equation,  the  inertial 
equation,  that  describes  the  flow  on  the  slow  manifold.  This  equation  has  half  the  dimension 
of  the  Maxey-Riley  equation;  this  fact  simplifies  both  the  qualitative  analysis  of  inertial 
dynamics  and  the  numerical  tracking  of  finite-size  particles. 


Singular  perturbation  formulation 

The  derivation  of  the  equation  of  motion  (6.2)  is  only  correct  under  the  assumption  /!>  1, 
which  motivates  us  to  introduce  the  small  parameter 

1 

e  =  -  «  1, 

and  rewrite  (6.2)  as  a  first-order  system  of  differential  equations: 


x  =  v. 


ev  =  u(x,  t)  —  v+e 


3 R  D u(x,  t) 
~2  Dt 


+  e  1 


(6.3) 


This  formulation  shows  that  x  is  a  slow  variable  changing  at  0(1)  speeds,  while  the  fast 
variable  v  varies  at  speeds  of  0(  1/e). 

To  transform  the  above  singular  perturbation  problem  to  a  regular  perturbation  prob¬ 
lem,  we  select  an  arbitrary  initial  time  to  and  introduce  the  fast  time  r  by  letting 


er  =  t  —  to- 

This  type  of  rescaling  is  standard  in  singular  perturbation  theory  with  to  =  0.  The  new 
feature  here  is  the  introduction  of  a  nonzero  present  time  to  about  which  we  introduce  the 
new  fast  time  r.  This  trick  enables  us  to  extend  existing  singular  perturbation  techniques 
to  unsteady  flows. 
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Denoting  differentiation  with  respect  to  r  by  prime,  we  rewrite  (6.3)  as 


x  =  ev, 

=  e, 

V  =  u(x,  (j))  —  V 

3 R  Du(x,  (j)) 


+  e- 


Dt 


.  ,  3i?\ 

+  Oi-T)g. 


(6.4) 


where  4>  =  fo+eT  is  a  dummy  variable  that  renders  the  above  system  of  differential  equations 
autonomous  in  the  variables  (x,  0,  v)  £  T>  x  IxR";  here  n  is  the  dimension  of  the  domain 
of  definition  V  of  the  fluid  flow  (n  =  2  for  planar  flows,  and  n  =  3  for  three-dimensional 
flows). 


Slow  manifold  and  inertial  equation 

The  e  =  0  limit  of  system  (6.4), 


x'  =  0,  (6.5) 

0'  =  0, 

v7  =  u(x»  -  V, 

has  an  n  +  1-paranreter  family  of  fixed  points  satisfying  v  =  u(x,  </>).  More  formally,  for 
any  time  T  >  0,  the  compact  invariant  set 

M0  =  {(x,</>,v)  :  v  =  u(x,0),  x  £  V,  4>  £  [t0  -  T,t0  +  T]} 

is  completely  filled  with  fixed  points  of  (6.5).  Note  that  Mq  is  a  graph  over  the  compact 
domain 

Do  =  {(x,</>)  :  x  £  V,  cf)  £  [t0  -  T,t0  +  T}}  ; 

we  show  the  geometry  of  Dq  and  Mq  in  Figure  6-1. 

Inspecting  the  Jacobian 

^  [u(x,  <f>)  -  v]Mo  =  -InXn, 

we  find  that  Mo  attracts  nearby  trajectories  at  a  uniform  exponential  rate  of  exp  (— r)  (i.e., 
exp  (— i/e)  in  terms  of  the  original  unsealed  time).  In  fact,  Md  attracts  all  the  solutions  of 
(6.5)  that  satisfy  (x(0),  </>(0))  £  Vx  [to  —  T,  to  +  T];  this  can  be  verified  using  the  last  equa¬ 
tion  of  (6.5),  which  is  explicitly  solvable  for  any  constant  value  of  x  and  cj).  Consequently, 
Mo  is  a  compact  normally  hyperbolic  invariant  set  that  has  an  open  domain  of  attraction. 
Note  that  Mo  is  not  a  manifold  because  its  boundary 

dMo  =  dVx  [to  -  T,t0  +  T]\jvx  {to  -  T}\jvx  {t0  +  T} 

has  corners;  Mq  —  8Mq .  however,  is  an  n  +  1-dimensional  normally  hyperbolic  invariant 
manifold. 

By  the  results  of  Fenichel  [43]  for  autonomous  systems,  any  compact  normally  hyperbolic 
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(a)  (h) 

Figure  6-1:  (a)  The  geometry  of  the  domain  Dq  (b)  The  attracting  set  of  fixed  points  Mo; 
each  point  p  in  Mq  has  a  n-dimensional  stable  manifold  fo(p)  (unperturbed  stable  fiber  at 
p)  satisfying  (x,  f>)  =  const. 


Figure  6-2:  (a)  The  geometry  of  the  slow  manifold  Me  (b)  A  trajectory  intersecting  a  stable 
fiber  /*  (p)  converges  to  the  trajectory  through  the  fiber  base  point  p. 


set  of  fixed  points  on  (6.5)  gives  rise  to  a  nearby  locally  invariant  manifold  for  system  (6.4). 
(Local  invariance  means  that  trajectories  can  only  leave  the  manifold  through  its  boundary.) 
In  our  context,  Fenichel’s  results  guarantee  the  existence  of  eo  (to,  T )  >  0,  such  that  for  all 
e  G  [0,  eo),  system  (6.4)  admits  an  attracting  locally  invariant  manifold  Me  that  is  O(e) 
CT-close  to  Mo  (See  Figure  6-2).  The  manifold  Me  can  be  written  in  the  form  of  a  Taylor 
expansion 

Me  =  {(x,  (j),  v)  :  v  =  u(x,0)  +  eu1(x,«^)  T  . . .  +  erur(x,0)  T  0(er+1),  (x,  (f>)  G  -D0}  ; 

(6.6) 

the  functions  u^(x,  (j>)  are  as  smooth  as  the  right-hand  side  of  (6.3).  Me  is  a  slow  manifold, 
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because  (6.4)  restricted  to  Me  is  a  slowly  varying  system  of  the  form 


x  =  ev 


Mt 


(6.7) 


=  e  [u(x,  +  eux(x,  0)  +  . . .  +  erur(x,  f>)  +  0(er+1)  . 


We  find  the  functions  ufc(x,  cf)  using  the  invariance  of  Me,  which  allows  us  to  differentiate 
the  equation  defining  Me  in  (6.6)  with  respect  to  r.  Specifically,  differentiating 


v  = 


u(x,  <j>)  +  ^2  ekuk(x,  <t>)  +  0(er+1) 


k= 1 


with  respect  to  r  gives 


v'  =  uxx'  +  U0(/>'  +  ^2  ek  u^x'  +  u^4>'  +  0(e 


r+l'i 


k=l 


on  Me,  while  restricting  the  v  equations  in  (6.3)  to  Me  gives 


v  = 


3R  Du  (  3 R\ 

u-v+e— — +  e  1-—  g 


2  Dt 


2  J 


J  Me 


Eh  h  f  /\  Oil  i/U  /  3  R\ 

elu‘(x»+e— — +  e(l-—  Jg 


3 R  Du 


k= 1 


(6.8) 


(6.9) 


Comparing  terms  containing  equal  powers  of  e  in  (6.8)  and  (6.9),  then  passing  back  to  the 
original  time  t,  we  obtain  the  following  result. 


Theorem  14  For  small  e  >  0,  the  equation  of  particle  motion  (6.7)  on  the  slow  manifold 
Me  can  be  rewritten  as 

x  =  u(x,  t)  +  eu1(x,  t)  +  . . .  +  erur(x,  t)  +  (D(er+1),  (6.10) 

where  r  is  an  arbitrary  but  finite  integer,  and  the  functions  u*(x,  t)  are  given  by 


,  .3  R 

u  = - 1 

2 


Hu 

~Dt 


~  g 


uk  =  - 


Du 


k- 1 


k—2 


Dt 


+  (Vu)  u*-1  +  (Vu‘ J  ufc_Z_1 


i= 1 


k  >  2. 


(6.11) 


We  shall  refer  to  (6.10)  with  the  u*(x,  t)  defined  in  (6.11)  as  the  inertial  equation  as¬ 
sociated  with  the  velocity  field  u(x,  t),  because  (6.10)  gives  the  general  asymptotic  form  of 
inertial  particle  motion  induced  by  u(x,t).  A  leading-order  approximation  to  the  inertial 
equations  is  given  by 


x  =  u(x,  t)  +  e 


D u(x,  t) 
Dt 


(6.12) 


this  is  the  lowest-order  truncation  of  (6.10)  that  has  nonzero  divergence,  and  hence  is  capable 
of  capturing  clustering  or  dispersion  arising  from  finite-size  effects. 
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Figure  6-3:  Sudden  changes  in  the  velocity  field  delay  convergence  to  the  slow  manifold. 


The  above  argument  renders  the  slow  manifold  Me  over  the  fixed  time  interval  [to  —  T,  to  +  T] . 
Since  the  choice  of  to  and  T  was  arbitrary,  we  can  extend  the  existence  result  of  Me  to  an 
arbitrary  long  finite  time  interval. 

Slow  manifolds  are  typically  not  unique,  but  obey  the  same  asymptotic  expansion  (6.11). 
Consequently,  any  two  slow  manifolds  and  the  corresponding  inertial  equations  are  0(er ) 
close  to  each  other.  Specifically,  if  r  =  oo,  then  the  difference  between  any  two  slow 
manifolds  is  exponentially  small  in  e.  The  case  of  neutrally  buoyant  particles  ( R  =  2/3) 
turns  out  to  be  special:  the  slow  manifold  is  the  unique  invariant  surface 

Me  =  {(x,0,  v)  :  v  =  u (x,0),  (x,  0)<EDO}, 

on  which  the  dynamics  coincides  with  those  of  infinitesimally  small  particles.  This  invariant 
surface  survives  for  arbitrary  e  >  0,  as  noticed  by  Babiano  et  al.  [8],  but  may  lose  its  stability 
for  larger  values  of  e. 

Convergence  to  the  slow  manifold 

The  results  of  Fenichel  [43]  guarantee  exponential  convergence  of  solutions  of  (6.4)  to  the 
slow  manifold  Me.  Translated  to  the  original  variables,  exponential  convergence  with  a 
uniform  exponent  to  the  slow  manifold  is  only  guaranteed  over  the  compact  time  interval 
[to  —  T,to  +  T]. 

Over  finite  time  intervals,  exponentially  dominated  convergence  is  not  necessarily  mono¬ 
tone.  For  instance,  if  the  velocity  field  suddenly  changes,  say,  at  speeds  comparable  to 
O  (1/e),  then  converged  solutions  may  suddenly  find  themselves  again  at  an  increased  dis¬ 
tance  from  the  slow  manifold  before  they  start  converging  again  (cf.  Figure  6-3).  Again, 
this  is  the  consequence  of  the  lack  of  compactness  in  time,  which  results  in  a  lack  of  uniform 
exponential  convergence  to  the  slow  manifold  over  infinite  times. 

Where  do  solutions  converging  to  the  slow  manifold  tend  asymptotically?  Observe  that 
for  e  =  0,  each  solution  converging  to  Mq  is  confined  to  an  n-dimensional  plane 

fo(p)  =  {(xp,0p,v)  :  P={  Xp,0p,u(xp,(/>p))  G  M0}  . 

Fenichel  refers  to  f\]  (p)  as  the  stable  fiber  associated  with  the  point  p:  each  trajectory  in 
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fo(p)  converges  to  the  base  point  of  the  fiber,  p.  More  generally,  a  stable  fiber  has  the 
property  that  each  solution  intersecting  the  fiber  converges  exponentially  in  time  to  the 
solution  passing  through  the  base  point  of  the  fiber.  The  collection  of  all  fibers  intersecting 
Mo  is  called  the  stable  foliation  of  Mo,  or  simply  the  stable  manifold  of  Mo. 

Fenichel  [43]  showed  that  the  stable  foliation  of  Mo  smoothly  persists  for  small  enough 
e  >  0.  Specifically,  associated  with  each  point  p  £  Me,  there  is  an  n-dinrensional  manifold 
ft (p)  such  that  any  solution  of  (6.4)  intersecting  ff(p)  will  converge  at  an  exponential  rate 
to  the  solution  that  runs  through  the  point  p  on  Me.  The  persisting  stable  fibers  ff(p)  are 
Cr  smooth  in  e,  hence  they  are  0(e )  CU-close  to  the  invariant  planes  fo(p),  as  indicated  in 
Figure  6-2b. 


6.2.2  Instabilities  on  the  dynamics  of  finite-size  particles 

As  we  saw  in  the  previous  subsection  the  velocity  of  a  finite-size  spherical  particle  typically 
differs  from  the  local  velocity  vector  of  the  ambient  fluid  flow.  In  particular,  we  saw  that  an 
exponentially  attracting  slow  manifold  exists  for  general  unsteady  inertial  particle  motion 
as  long  as  the  particle  Stokes  number  is  small  enough.  We  also  derived  an  explicit  reduced 
equation  on  the  slow  manifold  ( inertial  equation)  that  governs  the  asymptotic  behavior  of 
particles.  In  the  case  of  neutrally  buoyant  particles  (suspensions),  the  inertial  equation 
coincides  with  the  equations  of  Lagrangian  particle  motion.  This  would  seem  to  imply  that 
neutrally  buoyant  particles  should  synchronize  exponentially  fast  with  Lagrangian  particle 
dynamics  for  small  Stokes  numbers. 

By  contrast,  Babiano  et  al.  [8]  and  Vilela  et  al.  [152]  give  numerical  evidence  that 
two-dimensional  suspensions  do  not  approach  Lagrangian  particle  motions;  instead,  their 
trajectories  scatter  around  unstable  manifolds  of  the  Lagrangian  particle  dynamics.  Szeri 
et  al.  [139]  present  specific  examples  of  suspended  microstructures  in  two  dimensional 
fluid  flows  where  small  changes  of  the  modelling  assumptions  lead  to  drastically  different 
dynamics.  Babiano  et  al.  [8]  derive  a  criterion  that  characterizes  the  unstable  regions  in 
which  scattering  of  inertial  particles  occurs.  Their  derivation  follows  an  Okubo- Weiss-type 
heuristic  reasoning,  where  it  is  assumed  that  the  rate  of  change  of  the  velocity  gradient 
tensor  calculated  on  a  particle  trajectory  is  small  and  hence  can  be  neglected.  However,  as 
known  counterexamples  show  (cf.  Pierrehumbert  and  Yang  [113]  and  Boffetta  et  al.  [22]) 
such  reasoning,  in  general,  yields  incorrect  stability  results  except  near  fixed  points  of  the 
flow  field. 

As  we  saw  in  the  last  subsection  for  e  >  0  small  enough,  equation  (6.2)  admits  a  globally 
attracting  invariant  slow  manifold.  For  neutrally  buoyant  particles,  has  the  form 

=  {(x,  </>,  v)  :  v  =  u  (x,  (p)}  ;  (6.13) 

for  non-neutrally-buoyant  particles,  Me  is  given  by  a  graph  v  =  u  (x,  </>)  +  O(e). 

The  dynamics  on  Me  is  governed  by  the  reduced  Maxey- Riley  equation  ( inertial  equa¬ 
tion) 

x  =  u  (x,  t) ,  (6.14) 

i.e. ,  by  the  equation  of  motion  for  infinitesimal  fluid  elements. 

Using  an  observation  of  Babiano  et  al.  [8],  we  can  conclude  that  the  invariant  manifold 
Me  and  the  corresponding  reduced  equation  (6.14)  exists  for  all  values  of  e  in  the  neutrally 


127 


buoyant  case.  Specifically,  for  R  =  |,  equation  (6.3)  is  equivalent  to 

v  —  ut—  (Vu)  u  =  ~n  (v  —  u)  . 

As  pointed  out  by  Babiano  et  al.  [8],  this  last  equation  can  be  recast  in  the  form 
v  —  Ui—  (Vu)  v  =  —[i  (v  —  u)  +  (Vu)  (u  —  v) , 


or,  equivalently, 

[v  -  u  (x,  t)]  =  -  [Vu  (x,  t)  +  //I]  [v  -  u  (x,  t)}  ,  (6.15) 

d 

—  X  =  V. 

dt 

This  shows  that  Me  defined  in  (6.13)  is  an  invariant  manifold  for  any  fi  =  1/e. 


Global  attractivity  of  the  slow  manifold 

While  Me  exists  for  any  e  >  0,  it  is  not  guaranteed  to  be  globally  attracting  for  larger 
values  of  e  (i.e.,  for  smaller  values  of  fi).  Here  we  give  a  sufficient  condition  under  which 
the  globally  attractivity  of  Me  is  guaranteed. 


Theorem  15  Assume  that  for  some  fixed  e  >  0,  the  smallest  eigenvalue  field  Amjn(x,  t)  of 
the  symmetric  tensor  field  1  +  eS  (x,  t)  is  uniformly  positive  for  all  x  and  t  E  M+.  Then 
the  invariant  manifold  Me  is  globally  attracting,  i.e.,  all  neutrally  buoyant  particle  motions 
synchronize  exponentially  fast  with  infinitesimal  Lagrangian  fluid  trajectories. 


Proof:  See  Sapsis  and  Haller  2008  [128]. 

For  two-dimensional  incompressible  flows,  Amin  [I  +  eS]  is  the  smaller  root  of  the  char¬ 
acteristic  equation 

A2  —  2A  +  (l  +  e2detS)  =0. 

Therefore,  the  two-dimensional  version  of  the  sufficient  condition  in  the  above  theorem 
requires  that 

Amin  =  1  -  e^-detS  (x,  t)  >  0, 

or,  equivalently, 

fj,  —  y/det  S  (x,  t)\  >  0  (6.16) 

holds  uniformly  for  all  x  E  T>  and  i£l. 

Note  that  Theorem  15  may  be  generalized  for  the  case  of  non-neutrally  buoyant  particles 
by  studying  the  normal  stability  of  the  manifold  (6.6) 

Me  =  {(x,  (/>,  v)  :  v  =  u(x,0)  +  eu^x,^)  +  . . .  +  erur(x,<£)  T  0(er+1),  (x,  <f>)  E  D0}  . 


In  this  case  the  stability  condition  takes  the  form 


An 


S  (x(t),t)  +  ^1 


+  0(e)  <  0. 


(6.17) 


See  Appendix  C  for  details. 
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6.2.3  Clustering  of  finite-size  particles 

A  well-documented  phenomenon  displayed  by  inertial  particles  is  clustering,  i.e.,  concentra¬ 
tion  into  narrow  bands.  Several  studies  have  analyzed  inertial  particle  dynamics  in  either 
analytically  defined  or  numerically  generated  fluid  flows  (cf.  Maxey  and  Riley  [98],  Tang  et 
al.  [140],  Tio  et  al.  [145],  Marcu  et  al.  [94],  Martin  and  Meiburg  [95],  Marcu  and  Meiburg 
[93],  Vasiliev  and  Neishtadt  [151],  Rubin  et  al.  [121],  Crowe  at  al.  [32],  Burns  et  al.  [24]). 
These  studies  are  based  on  the  Maxey- Riley  equations  [98] ,  the  equation  of  motion  for  small 
spherical  particles  in  an  unsteady  non-uniform  flow  velocity  field. 

The  first  systematic  attempt  to  predict  particle  clustering  appears  to  be  by  Rubin  et 
al.  [121],  who  study  the  settling  of  aerosol  particles  in  a  two-dimensional  cellular  flow  field. 
Applying  results  of  singular  perturbation  theory,  they  show  the  existence  of  a  globally 
attracting  slow  manifold  to  which  inertial  particle  velocities  converge.  Reduction  to  the 
slow  manifold  coupled  with  a  subharmonic  Melnikov  calculation  reveals  that  particles  will  be 
attracted  to  (and  hence  cluster  around)  an  attracting  periodic  path  as  they  settle  downwards 
through  a  cellular  flow  field.  Sedimentation  patterns  for  Stokes  particles  in  a  weakly  time- 
periodic  flow  have  been  studied  by  Angilella  [4]  using  similar  methods. 

Burns  et  al.  [24]  investigate  the  motion  of  small,  dilute  spherical  particles  in  the  far 
wake  of  a  bluff  body  flow  model.  Using  the  approach  of  Rubin  et  al.  [121],  they  show 
numerically  the  existence  of  a  periodic  attractor,  i.e.,  the  location  of  clustering  in  the  wake. 
A  more  recent  numerical  study  by  Vilela  et  al.  [153]  visualizes  the  attractor  around  which 
heavy  particles  cluster  in  a  time-periodic  flow. 

The  objective  of  the  present  subsection  is  to  present  a  general  criterion  for  predicting 
inertial  particle  clustering  in  three-dimensional  steady  velocity  fields  of  the  form 

u  =(u(x),  u(x),  w(x.)),  x  =  ( x ,  y,  z)  £  M3 

and  in  two-dimensional  time-periodic  velocity  fields  of  the  form 

u  =(m(x),  u(x),  uj),  x  =  (x,  y,  (j>)  £  M2  x  S1.  (6.18) 

The  latter  velocity  field  is  also  represented  as  three-dimensional,  including  the  third  velocity 
component  (f>  =  co  in  the  three-dimensional  extended  phase  space  of  the  spatial  variables 
(x,y)  and  the  phase  variable  <j)  on  the  standard  unit  circle  S1. 

The  main  assumption  we  make  is  that  the  underlying  fluid  velocity  field  contains  at 
least  one  closed  two-dimensional  stream  surface.  This  certainly  holds  if  u  describes  a  3D 
steady  Euler  flow  in  which  the  Beltrami  condition  does  not  hold,  i.e.,  vorticity  and  velocity 
are  never  parallel.  In  this  case,  the  flow  is  integrable  and  the  flow  domain  is  foliated  by 
continuous  families  of  stream  surfaces  diffeomorphic  cylinders  or  tori  (Arnold  and  Keshin 
[6]).  Isolated  closed  stream  surfaces  will  also  exist  in  three-dimensional  nonintegrable  flows; 
a  well-known  example  is  the  non-integrable  Arnold-Beltrami-Childress  (ABC)  flow  which 
has  KAM  tori  (Arnold  and  Keshin  [6]).  Finally,  KAM  tori  typically  exist  in  two-dimensional, 
incompressible  flows;  their  signature  is  a  closed  invariant  curve  for  the  associated  Poincare 
map. 

Under  the  above  assumption,  we  use  the  inertial  equation  to  reduce  the  full  Maxey- 
Riley  dynamics  of  small  inertial  particles  to  a  three-dimensional  slow  manifold.  Given  the 
existence  of  a  closed  stream  surface  Sq,  we  derive  a  necessary  condition  that  guarantees  the 
existence  of  a  nearby  particle  attractor  on  the  slow  manifold.  The  criterion  requires  the 
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integral  of  the  normal  component  of  the  material  derivative 


Dt  =u.  +  (Vu)u 

over  So  to  vanish  for  a  nearby  attractor  Sf  to  exist  for  particles  of  mass  e.  The  vanish¬ 
ing  of  this  integral  on  So,  therefore,  predicts  clustering  on  a  nearby  surface  Se  which  is 
diffeonrorphic  to  Sq. 

For  the  special  case  when  the  fluid  particle  motion  is  dense  in  So,  we  use  ergodic  theory 
to  reformulate  our  clustering  criterion.  The  result  is  a  simplified  clustering  criterion  that 
only  requires  the  evaluation  of  a  line  integral  along  a  single  fluid  trajectory  in  Sq-  This  for¬ 
mulation  is  particularly  helpful  for  numerically  or  experimentally  generated  velocity  fields, 
where  exact  expressions  for  closed  stream  surfaces  are  not  readily  available,  but  individual 
fluid  trajectories  are  simple  to  generate. 


Formulation 


We  consider  the  case  of  three-dimensional  steady  flows,  and  finite  size  particles  for  which 
the  stability  criterion  (6.17)  is  everywhere  valid.  In  this  case  a  reduction  of  the  dynamics 
to  the  slow  manifold  Me  leads  to  the  inertial  equation 


x  =  u  (x)  +  e 


Du  (x) 
Dt 


+  0(e2). 


(6.19) 


The  above  equation  also  holds  for  unsteady  flows  with  the  appropriate  modifications.  Specif¬ 
ically,  for  a  two-dimensional  velocity  field  (u(x,y,  Xt),v(x,y,  Xt))  that  is  27r/A-periodic  in 
time,  we  let 

x  =  (x,y,ip),  u  =  («(x),i>(x),  A) ,  (6.20) 

and  observe  that  the  inertial  equation  (6.19)  remains  valid.  As  a  result,  particle  trajectories 
of  such  two-dimensional  flows  approach  asymptotically  the  trajectories  of  (6.19),  as  long  as 
e  is  small  enough  for  condition  (6.17)  to  hold  on  the  domain  of  interest. 


Necessary  condition  for  clustering 

For  the  setting  described  above,  we  have  the  following  main  result-a  necessary  condition-for 
the  location  of  inertial  particle  clustering. 

Assume  that  the  three-dimensional  vector  field  u  (defined  as  (6.20)  is  incompressible, 
i.e.,  has  zero  divergence  with  respect  to  its  arguments  x.  Also  assume  that  Sq  is  a  compact, 
two-dimensional  stream  surface  for  u.  Let  us  denote  the  outward  unit  normal  of  So  at  point 
x  by  n(x).  Then  the  following  hold: 


Theorem  16  Assume  that  the  three-dimensional  vector  field  u  (defined  as  (6.20)  is  incom¬ 
pressible,  i.e.,  has  zero  divergence  with  respect  to  its  arguments  x.  Also  assume  that  So  is 
a  compact,  two-dimensional  stream  surface  for  u.  Let  us  denote  the  outward  unit  normal 
of  So  at  point  x  by  n(x).  Then  the  following  hold: 

(i)  A  necessary  condition  for  the  existence  of  an  inertial  particle  attractor  Se  that  is 
O(e)  C1  -close  to  So  is  the  following: 


n  dA  =  0, 


(3 R  -  2) 


f 

Du 

/  V 

- •  n 

Iso 

[Dt  \ 

•  n 


dA  <  0. 


(6.21) 
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(ii)  Assume  that  So  is  a  two-dimensional  torus  filled  densely  with  the  trajectories  of  the 
system  x  =  u(x).  Let  £(t)  be  one  of  these  dense  trajectories  on  So-  Then  condition  (6.21) 
is  equivalent  to 


lim 

T— >oo 


3R  —  2 
T 


dt  =  0, 


dt  <  0. 


(6.22) 


Proof:  Sapsis  and  Haller,  2010  [124], 


6.3  Dynamics  of  finite-size  particles  in  stochastic  flows 

6.3.1  Stochastic  velocity  field 

We  will  now  study  the  motion  of  finite-size  particles  in  the  presence  of  flow  uncertainty. 
We  consider  a  random  field  u  (x,  t;  uj) ,  x  gD  C  Mn,  n  =  2, 3,  t  G  T,  uj  G  H  and  we  assume 
that  this  can  be  written  as  the  sum  of  a  mean  flow  U  (x,  t)  and  a  zero-mean  stochastic 
disturbance  for  which  we  utilize  the  stochastic  expansion  used  in  the  previous  chapters 

S 

u  (x,  i;  uj)  =  U  (x,  t)  +  (f ;  uj)  Uj  (x,  t) ,  uj  G  LI 

i= 1 

=  U  (x,  t )  +  Yi  (t;  uj)  u,;  (x,  t) ,  u j  G  Ll 

Moreover,  in  what  follows  we  will  denote  the  correlation  function  that  describes  their  second 
order  characteristics  as 


CYiYj  (h,h)  =  E“  [Yi  (; h-u )  Yj  (t2- uj)] 


Since  our  study  will  involve  inertial  particles  it  is  necessary  to  consider  the  acceleration  field 
given  by 


where 


and 


Du  (x,  t;uj) 
Dt 


(Vu  (x,  t;uj))  u  (x,  t;u)  +  ut  (x,  t\uj) 


D U  (x,  t) 
Dt 


+  a(x,i;w) 


DU  (x,  t) 
Dt 


Ut  (x,  t)  +  (VU  (x,  t))  U  (x,  t) 


(6.23) 


a  (x,  t\uS)  =  Yi  (f;  uj)  [(VU  (x,  t))  u?:  (x,  t)  +  (Vu*  (x,  t))  U  (x,  t)  +  u^t  (x,  t)] 

+  Yijt  (t\  uj)  Uj  (x,  t) 

+  Yi  (t;  uj)  Yj  (t-  uj)  (Vuj  (x,  t))  uj  (x,  t) 

with  Yijfifiuj)  =  and  Uj^  (x,  t)  =  .  Note  that  the  consideration  of  a  zero- 

mean  random  component  in  the  velocity  field  introduces  an  extra  non  zero-mean  term  in 
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the  acceleration  field.  Therefore  it  should  be  emphasized  that 


E“  [a  (x,  t;u)\  =  Cy iYj  (t,  t)  (Vu*  (x,  t))  uj  (x,  t) 


6.3.2  Markov  (diffusion)  approximation 

Inserting  representations  (3.1),  (6.23)  for  the  flow  velocity  and  acceleration  fields  into  the 
equation  of  motion  (6.3)  we  obtain 


3  R 


ev  —  e- 


D\J 

~Dt 


+  CYiYj  (t,  t )  (Vuj  (x,  t))  u j  (x,  t) 


where  C  ( x ,  t\u)  is  a  zero-mean  stochastic  process  defined  as 


3J? 

C  (x,  t;co)  =  Yi(t;u)  +  e—Yijt(t;u)) 
3  eR 


u i  (x,  t) 


=  -(v-U)  +  e(l  -  g  +  C  (x,  t;u) 

(6.24) 

(6.25) 


+  -y  [Y.  (t\ u)  Yj  (f;  u)  -  CY iYj  (t,  t)]  (Vu*  (x,  t))  uj  (x,  t) 

+  ~2~Yi  (t;  u)  [(VU  (x,  t))  Uj  (x,  t)  +  (Vu*  (x,  t))  U  (x,  t)  +  uijt  (x,  t)] 


Note  that  in  (6.24)  the  term  CyiYj  (t,  t)  (Vu*  (x,  t))  (x,  t)  can  also  be  written  as  Vx  (x,  y)  | 

Equation  (6.24)  describes  the  stochastic  dynamics  of  finite  size  particles  in  a  random  flow. 

The  zero-mean  stochastic  process  £  (x,  t]to)  has  in  general  finite  correlation  length  and  there¬ 
fore  (6.24)  is  not  an  Ito  Stochastic  differential  equation  for  which  many  analytical  tools  exist 
to  derive  equations  describing  the  probability  density  function  of  the  solution. 

In  what  follows  we  will  derive  an  Ito  SDE  that  approximates  the  dynamics  of  (6.24). 

This  derivation  will  be  based  on  the  assumption  of  small  correlation  time  length  for  the 
stochastic  process  Y  (i;u>)  (which  is  the  only  stochastic  ingredient  of  ( ( x,t;u ))  relative  to 
the  timescale  of  the  mean  flow  U  (x,  t)  ,  T.  As  correlation  time  length  of  the  stochastic 
process  Y  (t;  to)  we  define  the  time  ry  for  which 


OO 

ty  (t)  =  max  Tj  j  (t)  =  max  CY  Y  (t,  t)  /  Cyy  (t,  t  +  t)  dr  (6.26) 

hi  hi  1  *  J  3 

o 

This  quantity  is  a  measure  of  the  memory  of  the  stochastic  process  Y  (t\ lu)  .  For  stationary 
fields  the  correlation  time  length  is  independent  of  time.  For  the  general  case  we  will  assume 
that  the  statistical  characteristics  of  the  flow  are  slowly  varying  (with  respect  to  the  memory 
of  the  field  ty). 

For  the  case  where  ry  (t)  is  sufficiently  small  relative  to  the  deterministic  dynamics  time 
scale,  i.e.  when  ry  <  T,  Markov  (or  Diffusion)  approximation  can  be  applied  to  derive  an 
approximate  Ito  SDE  for  the  SDE  (6.24).  This  methodology  relies  on  the  approximation 
of  the  stochastic  process  ( (x,  t;u>)  by  a  process  with  independent  increments  with  respect 
to  time  (e.g.  Brownian  motion).  Therefore  even  if  the  process  £  (x,  t;ur)  may  not  posses 
the  property  of  independent  increments  (i.e.  zero  correlation  length)  if  the  deterministic 
dynamics  governing  the  evolution  of  x,  v  act  on  a  slower  time  scale  than  the  memory  of 
the  stochastic  process  (  then  the  independent  increment  approximation  is  valid  (Lin  &  Cai, 
1995  [86],  Klyatskin,  2005  [65]). 
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First  we  transform  the  above  stochastic  singular  perturbation  problem  to  a  regular 
stochastic  perturbation  problem  by  selecting  an  arbitrary  initial  time  to  and  introduce  the 
fast  time  r  by  letting 

£T  =  t  —  to- 

Then  equation  (6.24)  will  take  the  form 

v'  -  ^  +  CYiYi  &  (Vu*  (x’  <A)  ui  (x’  0)  =  -  (v  -  U)  (6.27) 

+  e  (f  -  y)  s  +  C(x,0;^) 

(j)'  =  e  (6.28) 

where  we  denote  with  prime  the  differentiation  with  respect  to  r,  and  (f>  =  to  +  er.  Then 
using  diffusion  approximation,  the  dynamics  can  be  described  by  the  following  Ito  SDE  (Lin 
&  Cai,  1995  [86],  Klyatskin,  2005  [65]) 

dv  (</>)  =  €  Y  +  CYiYi  (Vu*(x>^))ul  (x>0)  dr  (6-29) 

+  -(v-U)  +  e^l  -  y)  8  dr  +  E  (x,  v,  0)  dW  (r;w) 
where  in  the  above  there  is  no  drift  correction  since  this  is  vanishing: 

<f)-\-TY  U 

?i  {x,  V,  4>)  =  ^  J  duje ,UJ  —  yy~^ 0  (x  (g;  x,  V,  0) ,  s;a;)  d.s  =  0  (6.30) 

TY  i  i  Vj 


since  (x,  </>;a;)  is  independent  of  v.  Moreover,  E  (x,  v,  (j))  is  a  matrix  such  that 

4>+  TY  l/>+TY 

Sjfc  (x,  v,  ^>)  Ejfc  (x,  v,  0)  =  Y  J  du  J  E“  i(i(x{u-,x,v,(t>)  ,s;u)]ds 


with  x  (it;  x,  v,  <f>o)  being  the  solution  of  the  following  deterministic  system 


(6.31) 


d2x  3 R  \DXJ  1  f  dx  TT\  A  3 R\ 

dp  ~  e~  [of  +  Cy‘r >  <*’  (Vu‘ <*’ «)  u»  (v  =  -  (^  -  u)  +  e  (i  -  TJ  g 


(6.32) 


X  (<f>o)  =  x  and  ±  (<f>o)  =  v 


We  will  now  proceed  to  the  asymptotic  computation  of  the  diffusion  coefficient  with  respect 
to  the  correlation  time  length  Ty-  More  specifically  from  equation  (6.31)  we  may  obtain  the 
zero  order  approximation  for  the  diffusion  coefficient  by  first  noting  that 

Ty  — >  0  CyrYj  (t,  s )  — >  2 Dij  ( t )  5  (t  -  s) 
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where  (by  direct  integration  from  0  to  oo  we  obtain) 


oo 

Dij  (</>)  =  J CviYj  (4>,  4>  +  r)  dr.  (6.33) 

o 

The  term  D y  {4>)  expresses  the  statistical  dependence  of  the  field  u?  on  the  field  Uj  over 
time  and  it  can  be  seen  as  a  measure  of  the  intensity  of  the  flow  stochasticity.  For  the  case 
of  zero-correlation  time  length  all  this  statistical  dependence  is  concentrated  in  the  current 
time  instant  and  therefore  we  have  an  infinite  correlation  (Dirac  function).  However,  this 
limit  can  still  be  used  to  obtain  a  zero-order  approximation  for  the  the  diffusion  coefficient. 
Thus,  we  will  have  from  equations  (6.25)  and  (6.31)  for  the  diffusion  coefficient 


<t>+T Y  <t>+TY 


(x,  0)  Ejfc  (x,  </>)  =  J  du  J  ER  [Yt  ((/)-,  u)  Yk  (s;  u)  (x,  0)  uk}j  (x,  s)]  ds  +  O  (ea2) 

<f>  <t> 

<P+ty  4>+t-y 

=  ym  0^-  J  du  J  CYlYk  (<t>,  s)  uij  (x,  (j))  ukJ  (x,  s)ds  +  0  (ea2, 


=  2 Dik  (4>)  uiti  (x,  (f>)  ukJ  (x,  <f>)  +  0  {ea2,  ^) 

OO 

=  2  J Cu(,t)u(vHT)  (x,  x)  dr  +  O  {ea2, 


=  2S0^  +  O(e^,-F) 


2  TX\ 
T  J 


where  a2  =  maxCy,yt  (4>,  4>)  expresses  the  order  of  the  stochastic  terms.  The  above  ap¬ 
proximation  for  diffusion  coefficient  coincide  with  those  that  describe  the  motion  of  fluid 
elements  (zero-mass  particles)  in  the  limit  of  zero  correlation  length  (see  e.g.  [65]).  Hence, 
in  this  level  of  approximation  (weakly  stochastic  flow)  the  effect  of  particle  inertia  does  not 
enter  through  the  diffusion  term. 


Using  the  above  expression  for  the  diffusion  coefficient  we  obtain  the  equation  of  motion 
for  the  finite-size  particles  in  the  following  form 


dv  (<f>)  = 


3 eR  DU 

~2~15t 


-  (v  -  U)  +  e  1  - 


3  R 


g  +^YLCyiYj  ((j>,  <j>)  (Vuj  (x,  4>))  Uj  (x,  4 >) 


dr 


+  v/2£0  (x,  4>)  dW  ( T-io )  +  O  (e, 

Reverting  back  to  the  original  time  t  will  result  in 

dv  (t)  = 


3RDU  1.  TT,  . 

- (v  -  U)  +  1  - 

2  Dt  e  v  '  1 


3 R\  3 R„  .  ,  .  ,,  .  , 

g  +  -yC'y.y^  (t,  t)  (Vu*  (x,  t))  Uj  (x,  t) 


+  (x,  t)  dW  (f,u)  +  O  {a2,  ^ )  . 


2 

U 

T 


dt 

(6.34) 
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Note,  that  the  white  noise  term  scales  as 


dW  (t;w)  _  1  dW  (i;w) 
dr  \J~e  dt 


since  Eu  ylW  (r;o;)2J  =  dr,  and  Eu  |  dW  ( t\u )  J  =  dt  and  therefore  the  above  rescaling  is 
the  one  that  will  give  dr  =  edt. 

Examining  more  carefully  equation  (6.34),  and  comparing  it  with  its  deterministic  coun¬ 
terpart  (equation  (6.3))  we  see  the  addition  of  a  white  noise  component  but  also  a  new  term 
in  the  deterministic  part  of  the  equation.  To  understand  better  this  new  term  we  assume 
that  the  correlation  time  length  is  constant  among  different  stochastic  components  Yt.  Then, 
using  equation  (6.33)  as  well  as  the  definition  of  the  correlation  time  length  rtJ  (t)  (equation 
(6.26)),  we  obtain 


3  R 

~Y 


CYiYj  (t,t)  (Vuj  (x,f)) 


U; 


(x,  t) 


2TY  {t)  Dij  ^  <'X’  ^  ^  ^  ^ 

oo 

2r^{t)  j Vx  C“(-.0u(-,t+r)  (x>  y) \y=x  ^ 

o 


This  term  is  related  exclusively  to  the  inertial  dynamics  of  the  finite-size  particles  and  for 
sufficiently  small  correlation  time  length  its  effect  can  become  comparable  with  the  effect  of 
the  mean  flow  (or  even  more  intense).  Therefore,  consideration  of  random  fields  that  have 
zero  correlation  time  length  (i.e.  Kraichnan  fields  [70])  will  result  unbounded  acceleration 
on  the  right  hand  side  of  the  equation  of  motion,  except  of  the  case  of  aerosol  particles 
(R  =  0)  [12]. 


6.3.3  Stochastic  slow  manifold 


For  simplicity  we  will  assume  now  that  all  stochastic  components  Y]  have  the  same  corre¬ 
lation  time  length  ry  which  is  assumed  constant  or  sufficiently  slowly  varying.  Then  the 
equation  describing  the  motion  of  finite  size  particles  will  have  the  form 


dx  =  vdt 

,  ,'3R  DU 

av  = 


2  Dt 


3R\ 


3  R 


-  -  (v  -  U)  +  (  1  -  —  J  g  +  —CYiYj  (t,  t )  (Vu i  (x,  t))  u j  (x,  t) 


+  (X,  t)  dW  (f; u)  +  O  (u2, 


(6.35a) 

dt 

(6.35b) 


where  E0 ^o=j Cu(.,t)u(-,t+r)  (x^ x)  dr. 
o 

System  (6.35)  is  a  stochastic  singular  perturbation  problem  which  in  the  absence  of 
stochastic  excitation  and  in  the  limit  e  =  0  admits  a  globally  attracting  invariant  manifold 
(see  Section  1  of  current  chapter).  Therefore,  from  Berglund  and  Gentz,  2003  [17]  we  have 
the  existence  of  a  stochastic  slow  manifold 

Mi  =  {(x,  v)  |  v  =  ue  (x,t,  e;w)} 
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Figure  6-4:  Particle  velocity  (blue  solid  curve)  attracted  by  a  concentrated  layer  of  prob¬ 
ability,  a  stochastic  slow  manifold.  The  red  solid  curve  denotes  the  mean  of  the  manifold 
and  the  dotted  curves  define  the  local  spread  of  probability  around  the  mean. 


which  has  the  form  of  a  concentrated  probability  measure  around  the  deterministic  slow 
manifold  and  which  attract  solutions  of  the  stochastic  differential  equation  (6.35)  (Figure 
6-4).  In  what  follows  we  will  determine  the  second  order  stochastic  characteristics  of  the 
stochastic  slow  manifold,  i.e.  mean  and  variance. 


Mean  of  the  stochastic  slow  manifold 


To  determine  the  mean  of  the  stochastic  slow  manifold  we  use  the  invariance  property  of 
ue  (x,f,  e;w)  and  apply  the  mean  value  operator  to  equation  (6.35b).  The  last  term  vanishes 
because  of  the  non-anticipative  property  of  white-noise  [136].  Therefore  we  obtain 
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Expanding  in  the  same  way  we  did  in  the  proof  of  Theorem  14  the  mean  of  the  stochastic 
slow  manifold,  v  with  respect  to  e  we  obtain 
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Variance  of  the  stochastic  slow  manifold 

To  determine  the  variance  of  the  probability  measure  around  the  mean  value  of  the  stochas¬ 
tic  slow  manifold  we  consider  equation  (6.35b)  .with  v  =  ue  (u£  is  invariant  manifold  for 
(6.35b))  and  we  subtract  from  it  equation  (6.36).  Then  by  setting  £s  =  v  —  u7  we  obtain 
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Now,  from  equation  (6.38)  we  have 
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From  the  last  equation  we  obtain  the  zero-order  approximation  for  the  variance  of  the 
stochastic  slow  manifold 

£,s  (x^> e)  =  (x> *)  Eo  (x,  t)  +  O  (or2,  .  (6.39) 

The  last  equation  describes  the  local  variance  of  the  stochastic  slow  manifold. 

6.3.4  Stochastic  inertial  equation 

The  next  step  of  our  analysis  will  involve  the  formulation  of  a  reduced  order  stochastic 
differential  equation,  a  stochastic  inertial  equation,  that  will  describe  the  reduced  order 
dynamics  on  the  stochastic  slow  manifold.  Taking  into  account  equation  (6.37),  (6.39) 
and  the  fact  that  we  are  already  in  the  delta  correlated  regime  we  have  the  following  Ito 
stochastic  differential  equation  approximating  the  dynamics  of  the  full  slow  fast  system  for 
order  1  timescales  ([18]) 

cfx=Udt  +  e  (  1 - —  j  g — — —  dt  (6.40) 
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where  the  new  term  on  the  deterministic  part  of  the  right  hand  side  is  the  Wong-Zakai 
correction  ([159],  [86])  due  to  the  spatial  dependence  of  So  (x,i).  For  e  =  0  the  above 
inertial  equation  is  reduced  to  the  delta-correlated  approximation  for  the  description  of 
fluid  elements  in  random  flows  ([65]).  On  the  other  hand  for  zero  stochasticity  of  the  flow 
we  recover  the  corresponding  approximation  of  the  deterministic  inertial  equation  derived 
in  the  first  section. 

Analyzing  the  terms  on  the  right  hand  side  we  have  for  each  line:  i)  the  deterministic 
effect  of  particles  inertia,  ii)  the  coupled  effect  of  particles  inertia  and  flow  uncertainty, 
iii)  the  effect  of  the  random  flow.  Thus,  the  diffusion  coefficient  and  the  associated  drift 
correction,  in  this  level  of  approximation,  do  not  take  into  account  the  effect  of  particles 
inertia.  The  coupled  effect  of  particles  inertia  and  flow  stochasticity  is  expressed  through 
the  term  e  (4^  —  l)  (t,  t)  (Vu*  (x,  t))  Uj  (x,  t)  and  as  we  shall  see  in  the  applications 
it  plays  an  important  role  for  the  formation  of  clustering  regions  for  finite-size  particles 
different  than  those  predicted  by  the  mean  flow. 
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Using  this  equation  we  can  directly  obtain  a  transport  equation  for  the  probability 
density  function  describing  the  evolution  of  finite-size  tracers  in  a  random  field.  Specifically, 
the  stochastic  inertial  equation  is  equivalent  with  the  following  Fokker-Planck-Kolmogorov 
equation  [136]  describing  the  probability  density  function  or  the  concentration  c  (x,  t)  of 
finite-size  tracers 
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where  the  last  term  in  the  second  line  vanishes  due  to  incompressibility  of  u^.  (x,  t)  .  There¬ 
fore,  the  transport  equation  will  take  the  form 


dc  d 
dt  dxi 
d 


c{Uz  +  e{l-^ 


DUi  .  duk  i  (x,  t)  ' 

g — -  cYkYt  (t,  t)  — y- - u,m  (X,  i) 


Dt 


dxr 


^  Dq 

=  ~dx~-  (  (x,  t)  So Jk  (x,  t) 


dxi 


or  equivalently 

dc 

d 

|  _ 

dt 

dxi 

JL( 

dxi  1 

c  [  Ui  +  e  [  1  — 


3i? 


-DA  duk,i  (x,  f)  ' 

g - FTT  -  Cyfcy,  (t,  i) - s - (x,  i) 


Dt 


<9,XV 


y  /  Cuy.yKOy+r)  (x,x)  dr 


The  last  equation  describes  the  forward  evolution  of  the  concentration  for  finite-size  particles 
with  given  characteristics  e,  R  under  the  assumption  of  small  correlation  time  length  for  the 
stochasticity  of  the  velocity  field  relative  to  the  mean  field  dynamics. 
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6.3.5  Stochastic  source  inversion 

The  stochastic  inertial  equation  may  also  be  used  for  the  source-inversion  of  finite-size 
particles  in  the  presence  of  diffusion  or  flow  uncertainty.  More  specifically,  as  it  is  shown  in 
Haller  and  Sapsis,  2008  [56]  the  reduced  order  inertial  equation  for  deterministic  flows  can  be 
inverted  since  the  reduction  to  the  slow  manifold  does  not  allow  for  numerical  instabilities; 
this  is  not  the  case  if  we  use  the  full  set  of  dynamical  equations  (6.3)  since  the  strong 
attraction  to  the  invariant  slow  manifold  (in  forward  time)  will  cause  numerical  blow-ups  if 
we  try  to  solve  these  equations  backwards. 

This  is  also  the  case  for  stochastic  flows.  Equation  (6.40)  is  numerically  well  defined 
both  in  forward  and  backward  time  since  the  vector  field  on  the  right  hand  side  is  a  small 
deformation  of  passive  scalar  advection  due  to  the  inertial  terms  and  the  presence  of  noise. 
This  allows  us  to  obtain  the  initial  position  of  inertial  particles  with  some  uncertainty  due 
to  the  randomness  of  the  flow. 


6.4  Higher  order  Lagrangian  stochastic  models 


In  the  previous  section  we  derived  a  Lagrangian  stochastic  model  based  on  the  diffusion 
approximation  of  the  random  terms.  As  we  saw  in  the  first  level  approximation  inertia  does 
not  modify  the  effect  of  diffusivity.  Therefore,  inertial  particles  diffuse  in  the  same  manner 
as  fluid  particles.  This  is  not  the  case  for  the  deterministic  terms  of  the  inertial  equation 
where  the  effect  of  particles  finite-size  enters  explicitly.  The  expression  for  the  diffusion 
coefficient  was  derived  based  on  the  assumption  of  very  small  correlation  time  length  for 
the  stochastic  term,  which  allowed  us  to  apply  diffusion  approximation.  However,  other 
methods  may  also  be  used  for  the  description  of  the  diffusive  dynamics. 

Methods  which  are  based  on  the  well-mixed  condition  [144]  allows  us  to  derive  higher 
order  Largangian  models  for  the  description  of  the  diffusive  dynamics  of  fluid  elements.  In 
this  framework  it  is  assumed  that  the  Eulerian  velocity  that  the  particle  ‘feels’  evolves  in  a 
Markovian  manner.  Based  on  these  assumptions  Thomson  [144]  considered  models  of  fluid 
particle  trajectories  in  which  the  trajectories  in  the  (x,  v)  —space  are  Markovian,  continuous 
and  have  the  same  local  structure  as  a  process  with  independent  increments.  Such  processes 
can  be  represented  as  solutions  of  Ito  stochastic  differential  equations  (cf.  [138]).  Based  on 
the  same  arguments,  we  may  assume  that  the  evolution  of  inertial  particles  is  described  by 
the  stochastic  model 
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where  the  drift  and  diffusion  coefficients  f ,  F  are  defined  based  on  the  well-mixed  condition 
([144],  see  also  [20])  which  essentially  guarantees  that  the  produced  process  v  (f;  lu)  will  have 
at  every  time  instant  the  exact  same  statistics  with  the  Eulerian  velocity  field  u  (x,  t ;  lu)  at 
the  location  x  =  x  (t;  tu) ,  where  the  particle  is  currently  moving. 

The  main  difference  of  the  inertial  particle  model  (6.42)  with  (6.40)  is  the  addition  of 
more  memory  in  the  diffusive  dynamics,  by  increasing  the  order  of  the  model  (two  stochastic 
equations  instead  of  one).  However,  this  is  also  the  main  disadvantage  of  equation  (6.42) 
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since  the  formulation  of  a  transport  equation  for  the  concentration  field  of  inertial  particles 
will  lead  to  an  advection-diffusion  equation  involving  six  spatial  variables  (three  for  the 
position  and  three  for  the  velocity).  Therefore,  higher  order  inertial  particle  models  may 
only  be  solved  through  Monte-Carlo  simulation. 

A  final  remark  on  the  above  discussion  is  that  through  the  above  method  inertial  effects 
may  also  be  taken  into  account  in  the  diffusive  dynamics  if  one  applies  the  well-mixed 
condition  directly  to  the  statistics  of  the  Eulerian  field  £  (x,  t\u)  (see  equation  (6.25)  for 
definition)  instead  of  the  Eulerian  statistics  of  the  flow  field  u  (x,  t ;  lo)  (which  is  the  first 
level  of  approximation  for  the  diffusivity  of  inertial  tracers  as  shown  in  Section  6.3.2). 


6.5  Clustering  due  to  stochasticity  of  the  flow 


In  this  section  we  will  discuss  the  coupled  effect  of  flow  stochasticity  and  particles  inertia 
on  the  clustering  properties  of  finite-size  particles.  By  clustering  of  finite-size  particles  in 
random  flows  we  mean  the  formation  of  narrow  zones  in  the  physical  phase  space  where  the 
concentration  of  finite-size  particles  becomes  very  important.  In  Section  6.2.3  we  saw  that 
in  deterministic  flows  the  cause  for  the  formation  of  these  zones  is  the  dissipative  dynamics 
induced  by  the  order  e  terms  in  the  inertial  equation. 

For  the  case  of  random  flows,  as  we  saw  in  Section  6.3.4,  we  have  the  dissipative  term 
e(l  —  ^E)  [g— found  also  in  the  deterministic  analysis,  the  white  noise  term  with  the  as¬ 
sociated  drift  correction  'Sq  (x,  t )  VHo  (x>  t ),  but  also  the  new  term  e  (l  —  ^)  Cy^y.  (t,  t )  (Vu*  (x,  t ))  Uj  (x,  t ) 
which  expresses  (to  the  first  order)  the  coupled  effect  of  particles  inertial  and  flow  stochas¬ 
ticity.  The  combined  effect  of  the  deterministic  terms  on  the  right  hand  side  of  (6.40)  causes 
particles  to  be  attracted  by  critical  manifolds  in  the  flow  which  are,  in  general,  different 
by  those  of  the  mean  flow  alone.  On  the  other  hand  the  existence  of  noise  prevents  the 
particles  to  cluster  on  lower  dimensional  manifolds  (which  is  the  case  in  deterministic  flows) 
since  it  induces  diffusive  behavior.  The  result  is  particles  to  concentrate  smoothly  around 
the  critical  manifolds  defined  by  the  deterministic  part  of  the  right  hand  side  of  (6.40). 

In  what  follows  we  will  analyze  more  rigorously  this  behavior.  Before  we  proceed  to  this 
analysis  let  us  understand  better  the  physical  effect  of  the  new  term  e  (l  —  ^)  Cy;y-.  ( t ,  t )  (Vu*  (x,  t ))  Uj  (x,  t ) 
on  the  deterministic  part  of  the  inertial  equation  by  considering  the  following  simplified  case. 

Assume  we  have  a  zero  mean  random  flow  field  which  has  the  following  form 

u  (x,  t\  u)  =  Y  (£;  lo)  uo  (x,  t)  (6.43) 

where  Y  (f ;  lo)  is  a  zero  mean  Gaussian  process  and  uo  (x,  t )  is  a  deterministic  flow  field 
for  which  the  deterministic  clustering  criterion  presented  previously  can  be  applied.  Fur¬ 
thermore,  we  assume  that  there  is  a  manifold  So  in  the  physical  phase  space  such  that  the 
clustering  criterion  (in  the  following  form;  see  Sapsis  and  Haller  2010  [124]  for  details)  holds, 

[  V.^dV  =  0 
4lnt(S0)  Dt 

The  above  condition  can  also  be  written  as 


/int(So) 


V-  [(Vuq)  uq]  dV  =  0. 


An  interesting  feature  of  the  last  condition  is  its  independence  of  the  sign  of  the  flow. 
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Figure  6-5:  Clustering  manifold  is  independent  from  the  flow  direction. 


Additionally,  a  similar  argument  on  the  second  condition 
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which  defines  the  kind  of  the  particles  (bubbles  or  aerosols)  that  will  cluster  around  Sq, 
reveals  that  particles  (either  bubbles  or  aerosols)  will  cluster  around  So  independently  from 
the  direction  of  the  flow.  Therefore,  returning  to  the  stochastic  flow  field  (6.43),  we  conclude 
that  the  same  kind  of  clustering  (i.e.  same  kind  of  particles  and  same  location)  will  occur 
for  both  the  cases  where  Y  (t;  u)  is  positive  and  negative  (Figure  6-5). 

Thus,  a  zero  mean  stochastic  perturbation  on  the  flow  velocity  field  can  induce  non¬ 
zero  probability  for  particles  clustering,  assuming  that  the  geometry  of  the  perturbation 
has  a  suitable  spatial  form  to  cause  clustering.  This  is  exactly  the  effect  that  the  coupling 
term  e  (l  —  ^p)  Cy^-.  (t,  t )  (Vu,;  (x,  t))  uj  (x,  t)  expresses  in  the  inertial  equation.  It  induces 
clustering  of  particles  and  depends  on  the  covariance  of  the  stochastic  coefficients  and  the 
geometry  of  the  stochastic  perturbations. 

For  the  general  case  of  two-dimensional,  random,  periodic  flows  (or  three  dimensional 
random  time-independent  flows)  we  will  use  the  unified  setting  described  in  Section  6.2.3. 
For  this  case  we  first  consider  the  following  deterministic  system  (using  the  notation  defined 
in  (6.20)) 


x=  v  (x,  ip)  =  U  (x,  ip)-e(l-  CYiYj  (p,  V)  (Vuj  (x,  <p))  u j  (x,  <p)  (6.44) 
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The  last  equation  is  the  deterministic  part  of  the  stochastic  inertial  equation.  Theorem  5.1.6 
from  Berglund  and  Gentz  [18]  guarantees  that  an  attracting  manifold  for  a  deterministic 
system  such  as  (6.44)  will  persist  as  a  domain  of  high  concentration  in  the  presence  of 
additive  noise,  i.e.  for  the  full  inertial  stochastic  equation  (i.e.  including  the  white-noise 
term),  assuming  that  the  noise  intensity  does  not  exceed  in  size  (order  of  magnitude)  the 
deterministic  terms. 

Therefore,  it  is  sufficient  to  describe  the  clustering  regions  for  the  dissipative  deter¬ 
ministic  dynamical  system  (6.44).  This  can  be  done  with  direct  numerical  integration  of 
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equation  (6.44)  or  by  assuming  a  decomposition  of  the  vector  field  v  (x,  p )  in  the  form  of 
an  irrotational  and  incompressible  component.  Specifically,  if  we  assume  that  there  is  a 
potential  0  (x,  p)  and  a  streamfunction  ijj  (x,  p)  such  that 

v  (x,  p)  =  V  x  (x,  ip)  +  V</>  (x,  ip) 

then  we  will  have  ijj  ~  0(1)  (since  this  term  will  contain  the  effect  of  the  incompressible 
field  U  (x,  p))  while  <j)  ~  O  (e) .  Based  on  this  decomposition  we  are  able  to  apply  directly 
the  analytical  results  of  Theorem  16  and  predict  whether  an  invariant  manifold  for  the 
incompressible  flow  x  =  V  x  if)  (x,  p)  will  persist  as  a  clustering  manifold  for  equation  (6.44) 
and  thus  for  the  full  stochastic  inertial  equation. 

We  emphasize  that  the  invariant  manifolds  (if  those  exist)  for  the  incompressible  flow 
x  =  V  x  ip  (x,  p)  are  in  general  different  from  those  predicted  by  the  mean  field  equation 
x  =  U  (x,  p)  since  there  are  terms  that  constitute  v  (x,  p),  other  than  U  (x,  p),  which  are 
incompressible  and  therefore  they  contribute  to  the  streamfunction  if).  This  will  be  also 
illustrated  in  the  numerical  example  presented  in  the  following  section. 


6.6  Application:  Particles  in  the  double  gyre  flow 

To  illustrate  our  theoretical  findings  we  consider  the  stochastic  double  gyre  flow  analyzed 
in  Chapter  5.  Specifically,  we  consider  the  case  presented  in  Section  5.8.3  for  Re  =  25  with 
different  parameters.  After  computing  the  stochastic  flow  field  we  rescale  time  by  a  factor 
of  100.  The  dimensional  parameters  of  the  flow  are  presented  in  6.1. 

In  Figure  6-6  we  present  the  stochastic  flow  field.  Specifically,  the  mean  flow  field  U  (x,  t ) 
is  shown  in  the  left-top  plot  (the  colormap  indicates  the  vorticity  and  the  solid  curves  the 
streamlines  of  the  flow).  For  this  case  of  parameters  we  have  two  stochastic  perturbations 
Uj  (x,  t),  i  =  1,2  shown  in  the  lower  plots  along  with  the  corresponding  probability  density 
functions  of  the  stochastic  coefficients  Yi  (t'u).  The  variance  Ew  [Y?  (f;cj)]  of  each  of  the 
stochastic  perturbations  is  shown  in  the  right-top  plot  as  blue  solid  curves.  The  red  curve 

indicates  the  energy  of  the  mean  flow,  f  j  |U  (x,  t)\2  dx. 

We  will  study  the  motion  of  finite-size  particles  for  t  =  100  where  the  a  typical  stochastic 
perturbation  has  an  order  of  magnitude  of  10%  of  the  typical  amplitude  of  the  mean  flow; 
therefore  in  this  regime  we  expect  the  effect  of  ffow  uncertainty  to  be  comparable  with  the 
effect  of  inertia.  After  t  =  100  the  dynamics  of  the  mean  flow  have  converged  to  a  stationary 
regime,  while  the  most  energetic  stochastic  mode  has  converged  to  a  periodic  attractor  with 
period  of  oscillation  (T  ~  2)  comparable  with  the  time  scale  of  motion  for  the  finite-size 
particles.  Therefore,  this  is  a  case  of  unsteady  flow  with  non-st ationary  statistics. 

Note,  that  for  particles  having  size  e  =  10~2  or  smaller  the  stability  criterion  (6.17) 
is  satisfied  for  almost  every  flow  realization  uj,  everywhere  in  the  domain.  Therefore,  the 
reduction  to  the  stochastic  inertial  manifold  is  valid. 

6.6.1  Stochastic  slow  manifold  in  the  flow 

We  consider  heavy  particles  with  e  =  0.001  and  R  =  0.4.  In  Figure  6-7  we  present  the 
modulus  of  the  velocity  of  the  slow  manifold  |u7  (x,f ,  e)  |  with  respect  to  the  horizontal 
dimensions  x  and  y  and  for  t  =  100.  In  this  time  regime  the  mean  flow  has  converged  to  a 
steady  form  even  though  the  stochastic  perturbations  perform  a  periodic  oscillation.  This 
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Table  6.1:  Reference  values  of  parameters  in  the  barotropic  QG  model  used  for  the  study 
of  finite-size  particles  (dimensional). 


Parameter 

Value 

Parameter 

Value 

U 

7.1  x  10_1m.s_1 

L 

1.0  x  106?n 

L/U 

16.27 days 

D 

2500m 

TO 

1.26  x  10_1Pa 

Po 

7.1  x  10”12  (m.sy1 

P 

10  3Kg.m~3 

fo 

5.0  x  10~5s-1 

Mean  flow  Time  -100 


Figure  6-6:  Stochastic  double  gyre  for  the  illustration  of  the  inertial  particles  motion; 
Re  =  25  (See  chapter  5  for  details). 
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i  o  y 


Figure  6-7:  Mean  value  of  the  stochastic  slow  manifold  governing  the  motion  of  inertial 
particles  for  the  double  gyre  stochastic  flow. 


is  clearly  seen  from  the  top-right  plot  where  we  present  the  energy  of  the  mean  flow  (red 
curve). 

In  Figures  6-8  and  6-9  we  show  the  spatial  distribution  of  uncertainty  in  the  stochastic 
slow  manifold  by  plotting  the  quantity 

OO 

=  7 - f FI - T  /  llCu(-,bu(-,t+r)  (x,  x)  ||  dr 

trace  |CY(t)Y(t)J  J 

for  different  time  instants,  where  ||-||  denotes  the  Euclidean  matrix  norm.  The  above  quan¬ 
tity  is  a  weighted  average  of  uncertainty  of  the  slow  manifold  due  to  stochasticity  coming 
from  different  modes.  The  total  variance  of  the  slow  manifold,  at  every  time  instant  can  be 
characterized  by  the  global  quantity  trace  [Cy(t)Y(t)]  which  is  shown  in  the  top-right  plot 
as  a  function  of  time.  In  the  considered  time  regime  the  variance  of  the  slow  manifold  is  of 
order  10-2  of  the  energy  of  the  mean  flow.  Therefore,  a  stochastic  fluctuation  will  have  a 
typical  magnitude  of  10%  relative  to  the  mean  flow. 

The  mean  value  of  the  stochastic  slow  manifold  depends  on  both  the  mean  flow  and 
the  stochastic  fluctuations.  However,  since  the  energy  of  the  mean  flow  is  an  order  of 
magnitude  higher  than  the  stochastic  fluctuations,  the  mean  value  of  the  stochastic  slow 
manifold  appears  almost  stationary.  On  the  other  hand,  the  typical  deviations  of  particle 
velocities  from  the  mean  slow  manifold  depend  strongly  on  both  space  and  time.  Therefore, 
in  accordance  with  the  plots  6-8  and  6-9  we  expect  higher  deviations  from  the  mean  value 
of  the  stochastic  slow  manifold  in  locations  and  time  instants  where  as  (x,  t )  is  higher. 
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t  =  100.0 


Variance  of  stochastic  SM 


Figure  6-8:  Variance  of  the  stochastic  slow  manifold  descrbing  of  motion  inertial  particles 
for  t  =  100.0  and  t  =  100.5. 
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t  =  101.0 


Variance  of  stochastic  SM 


Figure  6-9:  Variance  of  the  stochastic  slow  manifold  descrbing  of  motion  inertial  particles 
for  t  =  101.0  and  t  =  101.5. 
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6.6.2  Convergence  to  the  stochastic  slow  manifold 

We  shall  now  illustrate  the  convergence  of  the  stochastic  dynamics  in  the  concentrated 
layer  of  probability  that  defines  the  stochastic  slow  manifold.  We  consider  a  specific  flow 
realization  Ui  and  we  solve  the  deterministic  Maxey-Riley  equation  with  very  high  accuracy. 
We  initiate  an  inertial  particle  with  e  =  10-3  and  R  =  0.4  with  initial  velocity  away  from 
the  stochastic  manifold. 

In  Figure  6- 10a  we  present  the  distance  from  the  stochastic  slow  manifold 

z(t)  =  | E“  [u£  (x,t,e;w)]  -  v  (t;uji)  \ 

with  respect  to  the  horizontal  position  of  the  particle.  The  black  dot  indicates  the  projection 
of  the  particle  on  the  mean  of  the  stochastic  slow  manifold.  The  colormap  shows  the 
instantaneous  local  variance  of  the  stochastic  slow  manifold,  as  (x,  t).  In  Figure  6-10b  we 
present  with  blue  solid  curve  the  x— component  of  the  particle  velocity.  The  red  curves  show 
the  typical  spread  (two  typical  deviations)  of  probability  around  the  mean  of  the  stochastic 
slow  manifold.  Specifically,  they  are  defined  by  the  quantity 

OO 

ux,±2a(t)  =  ue(x(t)  ,t,e;u)  ±  - - j- - T  /  ||Cu(  tw  t+T)  (x,x)||  dr 

trace  [CY(t)Y(t)J  J 

and  they  illustrate  the  location  where  most  of  the  probability  measure  is  concentrated.  In 
Figure  6-10c  we  present  directly  the  distance  from  the  slow  manifold, 

zx(t)  =  vx  ( t;cji )  -  E “  [uex  (x,i,  e;a>)] 


together  with  the  curves 


OO 

zx,±2*(t)  =  ±~ - f-; - t  /  ||CU(. ,t)u(-,t+T)  (x,x)||  dr. 

trace  [CY(t)Y(t)J  J 

As  we  are  able  to  observe,  during  the  initial  phase  of  motion  we  have  rapid  convergence 
into  the  layer  of  probability  that  defines  the  stochastic  slow  manifold.  Subsequently,  the 
particle  continue  to  move  inside  the  stochastic  slow  manifold,  with  its  velocity  fluctuating 
(see  Figure  6-11).  Note  that  the  magnitude  of  the  jumps  from  the  mean  of  the  stochastic 
slow  manifold  are  in  full  accordance  with  the  bounds  defined  by  the  curves  zX:±2a(t)  (Figure 
6-1  lc). 

6.6.3  Validation  of  stochastic  inertial  equation 

The  next  step  of  our  analysis  involves  the  validation  of  the  reduced  order,  stochastic,  inertial 
equation  (6.40).  We  consider  an  initial  distribution  of  inertial  particles  (heavy  particles  with 
e  =  10-3  and  R  =  0.4)  that  is  described  by  a  Gaussian  probability  density  function  (see 
Figure  6-12-top  plot).  We  first  create  a  large  number  (N  =  104)  of  particles  positions  that 
follow  the  initial  Gaussian  distribution.  We  advect  those  particles  by  solving  the  stochastic 
inertial  equation  (6.40).  For  the  numerical  solution  of  the  stochastic  differential  equation 
we  use  the  Euler-Maruyama  method  (see  e.g.  [58],  [49]).  Then  we  advect  the  initial  sample 
set  of  particles  (which  approximate  the  initial  distribution)  using  the  deterministic  inertial 
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t=  100.115 


Figure  6-10:  (a)  Rapid  convergence  to  the  stochastic  slow  manifold  during  the  initial  phase  of 
motion.  The  colormap  denotes  the  local  variance  of  the  slow  manifold  as  (x,  t)  .  The  vertical 
coordinate  shows  the  distance  of  the  stochastic  dynamics  from  the  mean  slow  manifold,  (b) 
x—  component  of  the  particle  velocity,  resolved  according  to  Maxey-Riley  equation  for  a 
particular  flow  realization  (blue  solid  curve).  The  red  lines  indicate  the  local  spread  of 
probability  around  the  mean  slow  manifold  at  the  particle’s  location,  (c)  same  as  (b)  but 
now  the  distance  from  the  slow  manifold  is  shown. 
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t=  104.000 


(b) 


\A4A- 


(C) 


"lOO  101  102  103  104 

time 


Figure  6-11:  Same  as  6-10  but  for  a  later  time  instant. 
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equation  for  M  =  500  realizations  of  the  random  flow  field. 

The  results  are  shown  in  Figures  6-12,  6-13  for  5  time  instants.  In  the  left  column 
we  present  the  results  from  simulating  the  stochastic  inertial  equation,  while  in  the  right 
column  we  show  the  Monte-Carlo  simulation.  Note  that  the  computational  time  required 
for  the  Monte-Carlo  simulation  was  almost  500  times  higher  since  we  had  to  advect  the 
probability  density  function  for  every  flow  realization.  As  we  are  able  to  observe  the  full 
Monte-Carlo  simulation  is  slightly  more  diffusive  relative  to  the  stochastic  inertial  equation. 
However,  the  results  compare  satisfactory  even  for  larger  times. 

6.6.4  Clustering  due  to  the  combined  effect  of  inertial  and  flow  stochas- 
ticity 

We  will  now  comment  on  the  effect  of  flow  stochasticity  on  the  clustering  properties  of 
heavy  particles.  In  Figure  6-14  we  present  the  particles  concentration  (particles  have  the 
same  parameters  as  in  the  previous  section)  for  two  different  time  instants.  The  blue 
lines  represent  the  streamlines  for  the  mean  flow  U  (x,  t)  and  the  red  curves  represent  the 
clustering  manifolds  predicted  by  the  deterministic  part  of  the  stochastic  inertial  equation 
(6.44). 

During  the  initial  phase  of  motion,  the  initially  Gaussian  blob  is  transported  according 
to  the  mean  flow.  Additionally,  we  observe  a  stretching,  normal  to  the  mean  flow,  caused  by 
the  existence  of  the  stochastic  perturbation  (Figure  6-14  top).  As  we  are  able  to  observe  the 
stretching  of  the  probability  density  function,  i.e.  concentration  field  occurs  in  accordance 
with  the  clustering  manifolds  for  the  deterministic  part  of  the  stochastic  inertial  equation 
(red  curves). 

In  a  later  time  instant  the  concentration  field  has  been  transported  by  the  mean  flow 
while  it  has  also  aligned  according  to  the  clustering  manifolds  of  the  stochastic  perturbation 
(Figure  6-14  bottom).  We  observe  the  formation  of  a  closed  ring  of  particles  (on  the  right  of 
the  plot)  which  is  not  consistent  with  the  mean  flow  but  it  is  fully  justified  by  the  clustering 
regions  of  the  full  stochastic  flow. 

Therefore,  zero  mean  stochastic  perturbations,  even  of  low  intensity  (in  this  case  the 
variance  of  the  perturbation  is  1%  of  the  mean  flow  energy),  are  capable  to  influence  dras¬ 
tically,  the  evolution  of  the  concentration  field.  Through  the  developed  framework  we  have 
identified  the  geometry  of  these  deformations  and  we  have  correlate  their  form  with  the 
characteristics  of  the  stochastic  part  of  the  flow. 
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Stochastic  Inertial  Particles  Model 


t=  100.00 


Monte  Carlo  Simulation 


t  =  101.00 


102.00 


Figure  6-12:  Comparison  of  stochastic  inertial  equation  (6.40)  and  direct  Monte-Carlo  sim¬ 
ulation  for  t  =  100, 101, 102. 
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Stochastic  Inertial  Particles  Model 


t=  103.00 


Monte  Carlo  Simulation 


\  X 


t  =  104.00 
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t  =  100.4 


Figure  6-14:  Concentration  field  for  finite  size  particles  (heavy  particles)  for  two  different 
time  instants.  The  blue  lines  indicate  streamlines  for  the  mean  flow.  The  red  lines  indicate 
clustering  manifolds  for  the  stochastic  flow  field. 
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Chapter  7 

Conclusions 


In  this  chapter  we  will  summarize  our  contributions  and  results  obtained  within  the  context 
of  this  Ph.D.  thesis. 

Dynamically  orthogonal  field  equations:  We  have  derived  an  exact,  closed  set  of  equa¬ 
tions  that  determine  the  evolution  of  continuous  stochastic  fields  described  by  a  SPDE. 
By  hypothesizing  a  finite  order  dynamical  expansion  (DO  expansion)  we  derived  directly 
from  the  original  SPDE  a  system  of  differential  equations  consisting  of  a  PDE  for  the  mean 
field,  a  family  of  PDEs  for  the  orthonormal  basis  where  the  stochasticity  ‘lives’  as  well  as  a 
system  of  SDEs  that  describes  how  the  stochasticity  evolves  in  the  time  varying  stochastic 
subspace.  Therefore,  we  do  not  assume  an  a  priori  representation  neither  for  the  stochastic 
coefficients,  nor  for  the  spatial  structure  of  the  solution;  all  this  information  is  obtained 
directly  by  the  system  equations,  boundary  and  initial  conditions.  We  have  also  illustrated 
that  under  appropriate  assumptions  the  DO  approach  generates  both  the  equations  obtained 
by  the  Proper-Orthogonal-Decomposition  method  and  by  the  Polynomial-Chaos  method; 
thus  it  unifies  the  two  methodologies.  Even  though  the  developed  framework  is  valid  for 
non-snrooth  systems  its  efficient  numerical  properties  are  limited  by  the  order  of  the  non¬ 
linearity  (infinite  for  non-snrooth  systems),  similarly  with  the  other  analytical  approaches 
(PC  or  POD  method).  The  above  results  have  been  published  in  the  article  Sapsis  and 
Lermusiaux,  Physica  D,  2009  [129]. 

Adaptive  criteria  for  stochastic  dimensionality:  To  treat  the  strongly  transient  responses 
characterizing  complex  systems  such  as  those  found  in  oceanic  applications  we  have  devel¬ 
oped  a  set  of  adaptive  criteria  that  controls  the  size  of  the  stochastic  subspace  where  the 
reduction  of  the  full  dynamics  is  performed  in  the  context  of  the  DO  method.  Those  criteria 
are  based  on  the  covariance  operator  of  the  system  response  as  well  as  on  the  stochastic 
operator  expressing  the  dynamics  of  the  system.  The  above  material  is  included  in  Sapsis 
and  Lermusiaux,  2010  [126]. 

Computational  code  for  2D  stochastic  Navier- Stokes:  We  have  developed  and  validated  a 
computational  code  in  Matlab  that  implements  the  DO  field  equations  for  2D  Navier-Stokes 
in  general  geometries.  This  code  is  based  on  the  finite-differences  numerical  scheme  and 
also  incorporates  the  adaptive  criteria  for  the  stochastic  dimensionality  of  the  solution.  We 
used  this  code  for  the  validation  of  DO  methodology  by  solving  the  stochastic  fluid  flow  in  a 
cavity  and  behind  a  cylinder  (material  included  in  Sapsis  and  Lermusiaux,  Physica  D,  2009 
[129]).  This  code  also  served  as  a  basis  for  the  development  of  a  more  accurate  and  optimized 
version  based  on  the  finite- volume  approach  (in  collaboration  with  Mr.  Matt  Ueckermann). 
Using  the  finite-volume  code  we  obtained  the  stochastic  response  of  an  idealized  ‘double 
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gyre’  model,  which  has  elements  of  ocean,  atmospheric  and  climate  instability  behaviors, 
and  we  illustrated  its  response  over  different  dynamical  regimes. 

Dynamics  of  finite-size  particles  with  applications:  The  derivation  of  the  Lagrangian 
stochastic  model  presented  in  Chapter  6  was  based  on  our  own  work  on  the  deterministic 
dynamics  of  finite-size  particles.  More  specifically,  for  finite-size  particles  in  deterministic 
flows  we  derived  a  reduced  order  inertial  equations  that  describes  accurately  the  dynamics 
of  finite-size  particles  for  sufficiently  small  size  (Haller  and  Sapsis,  Physica  D,  2008  [56]). 
These  results  were  also  extended  and  applied  to  large  scale  geophysical  flows  in  Sapsis  and 
Haller,  J.  Atm.  Sc.,  2009  [123].  Dispersion  properties  of  finite-size  particles  were  studied 
analytically  in  Sapsis  and  Haller,  Physics  of  Fluids,  2008  [128]  and  Haller  and  Sapsis,  SIAM 
J.  Appl.  Dyn.  Syst.,  2010  [57].  Also,  analytical  criteria  were  derived  for  the  description  of 
clustering  regions  for  finite-size  particles  in  Sapsis  and  Haller,  Chaos,  2010  [124]. 

Lagrangian  Stochastic  transport  model  for  finite-size  particles:  Based  on  the  determin¬ 
istic  analysis  for  finite-size  particles  as  well  as  on  our  DO  representation  of  the  stochastic 
flow,  we  derived  a  reduced  order  Lagrangian  stochastic  model  describing  the  motion  of 
finite-size  particles  in  arbitrary  random  flows.  We  validated  this  model  for  a  specific  ran¬ 
dom  flow  through  the  comparison  with  direct  Monte-Carlo  simulations.  Also,  using  the 
reduced  order  stochastic  inertial  equation  we  illustrate  the  combined  effect  of  inertia  and 
flow  uncertainty  on  the  clustering  properties  of  finite-size  particles. 


7.1  Future  directions 

Future  directions  include  the  application  of  the  developed  framework  to  complex  systems 
characterized  by  uncertainty,  others  than  those  considered  in  the  context  of  this  thesis. 
Applications  of  this  kind  include  for  example  the  analysis  and  identification  of  structural 
systems  subjected  to  random  excitations  (e.g.  water  waves),  wave  propagation  in  random 
media,  as  well  as  biological  systems  (such  as  population  models)  in  environments  with 
uncertainty. 

Based  on  the  work  related  to  the  idealized  climate  model  an  interesting  direction  would 
be  the  analysis  of  the  effect  of  stochasticity  directly  introduced  from  the  external  forcing 
which  in  this  study  was  modeled  as  deterministic.  Another  promising  research  direction 
will  be  the  stochastic  analysis  of  the  flow  transition  from  deterministic  to  stochastic  states 
characterized  by  a  large  number  of  modes  (turbulence).  The  study  of  energy  flow  among 
the  modes  and  between  the  mean  field,  in  combination  with  the  DO  formulation  may  be 
used  for  the  development  of  reduced  order  dynamical  systems  that  will  contain  the  essential 
dynamics  that  characterize  this  transition. 

DO  may  also  be  used  for  characterization  and  prediction  of  more  realistic  systems 
(oceanic  and  atmospheric  flows).  For  realistic  numerical  studies,  non- intrusive  schemes 
(e.g.  [90],  [103])  based  on  our  DO  expansion  should  also  be  investigated.  Research  is  un¬ 
derway  for  developing  DO  schemes  for  data  assimilation  [80]  or  Bayesian  inference  [96]  in 
idealized  and  realistic  dynamical  systems.  Adaptive  modeling  and  adaptive  sampling  are 
also  promising  directions  (e.g.  [78],  [110]). 

Related  to  the  work  on  finite-size  particles  an  interesting  future  direction  is  the  devel¬ 
opment  of  transport  models  that  take  into  account,  apart  of  the  finite-size,  the  interaction 
between  finite-size  particles  due  to  clustering  that  occurs  because  of  inertia  (as  it  is  de¬ 
scribed  in  the  current  work).  Development  of  such  models  will  play  a  very  important  role 
on  the  understanding  of  cloud  formation  and  rain  initiation  (where  both  the  particles  finite- 


156 


size  and  their  interaction  play  an  important  role)  [114],  [41],  [131]  as  well  as  in  the  problem 
of  coexistence  between  several  species  of  plankton  in  the  hydrosphere  [119],  [84].  Another 
potential  application  of  interest  is  planet  formation  by  dust  accretion  in  the  solar  system 
[155],  [36], 

Research  is  underway  for  the  experimental  validation  of  the  developed  models  in  chaotic 
fluid  flows  using  PIV  techniques  [108],  [107].  To  this  end  an  interesting  direction  for  future 
research  will  be  the  extension  of  the  present  theory  in  more  complex  models  describing 
particles  motion  such  as  dynamical  equations  that  include  the  effect  of  non-spherical  shapes, 
take  into  account  memory  effects  (Basset-Boussinesq  terms),  and/or  contain  non-Stokes  (i.e. 
non-linear)  drag  forces. 
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Appendix  A 


DO  equations  for  3D  Navier-Stokes 
in  component  wise  form 


In  this  appendix  we  will  present  in  detail  the  DO  field  equations  for  the  special  case  of  fluid 
flows  governed  by  stochastic  Navier-Stokes.  All  the  results  will  be  presented  in  component¬ 
wise  form  which  is  most  suitable  the  development  of  computational  code.  The  general 
equations,  for  an  incompressible  fluid  in  a  domain  D  have  the  form 


du  dp  1  A  d  ( u 2)  d  (uv)  d  ( uw )  „  .  ,  .  , 

to  =  ~ai  +  ite' A"  -  ~8i - By - aT  +  fv  +  T* (x' ()  +  ^  (x’t;“) 

=  £u  [x  (x,  i;  u) ;  u>] 

dv  dp  1  .  d  (vu)  d  ( v 2)  d  (vw)  ,  .  .  .  . 

to  =  -By  +  rA”  ^  ~di - By - ^  - /«  +  ^ (*, t)  +  M*. < I") 

=  ^  [x  (x,t;<j)  ;w] 

<9n  1  A  d  (wu)  d(wv)  d  (w2) 

aF  =  -&  +  fcA,"--ki--k - +  + 

=  Cw  [x  (x,  t;  u) ;  u] 
dp  d  (up)  d  (vp)  d  ( wp ) 
dl  ^ 


dx 


dy 


dz 


+  KAp  =  Cp  \x  (x,  t;  uj) ;  w] 


du  dv  dw 
dx  +  dy^  dz 


where 

X  =  (u  (x,  t;  w) ,  u  (x,  t;  a;}  %w  (x,  i;  a;)  ,  p  (x,  t;  w)) 

is  the  state  vector  with  (u(x,t;u)  ,v(x,t;u) ,  w  (x,  t\  u))  being  the  flow  velocity  field  and 
p  (x,  t;  uj)  the  density.  The  pressure  field  is  denoted  with  p(x,t;uj),  f  =  fo  +  (3oy  is  the 
Coriolis  coefficient,  (tx  (x,  t) ,  ry  (x,  t)  ,  tz  (x,  t))  is  the  external  deterministic  stress  acting  on 
the  fluid,  and  (<px  (x,  t,\ uj)  ,  ipy  (x,  f;  uj)  ,  <pz  (x,  f;  us))  is  the  zero-mean  stochastic  component 
of  the  the  stress  for  which  we  assume  known  the  complete  statistical  information.  In  what 
follows  we  will  use  the  DO  field  equations  derived  in  the  previous  chapter  with  inner  product 


(Xl,X2) 


[u\U2  +  V\V2  +  W\W2  +  P1P2]  dx.. 


D 
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We  will  also  use  the  notation 


{a  1,0:2} 


J ai  (x)  «2  (x)  cix. 
D 


By  performing  a  Karhunen-Loeve  expansion  on  the  stochastic  field  p  (x,  t;  uj)  we  may  ap¬ 
proximate  it  as 


R 

p  (x,  t;  w)  =  (f;  a;)  (x,  t.)  =  Zr  (f;  w)  pr  (x,  f) , 

r=l 

where  R  is  defined  by  the  order  of  truncation  of  the  KL  series  for  the  field  p  (x,  t ;  u>). 

For  simplicity  we  assume  that  the  boundary  conditions  are  either  deterministic  Neumann 
or  Dirichlet  for  the  velocity  field  and  Neumann  for  the  density  field 


u 

3u 

<9n 

dp 

On 


=  u<9Di  (£,t)  , 

{ CdD 1 

(€,t  ;uj) 

=  h u,8D2  (£$)  j 

£  G3D2 

(£,t;uj) 

=  hp,QD  (€,t)  , 

£  G9D. 

As  it  is  shown  in  chapter  5  the  case  of  stochastic  boundary  conditions  can  always  be  trans¬ 
formed  into  a  problem  with  deterministic  boundary  conditions  and  suitable  stochastic  forc¬ 
ing. 


Furthermore,  we  assume  that  the  initial  conditions  are  known  and  given  by 

u  (x,  to;  uj)  =  uo  (x;  u>) ,  xeD,  well 
p(x,  to;  w)  =  po  (x;  w) ,  xeD,  uj  e  Q. 


We  will  first  calculate  the  stochastic  operator  C.  By  using  the  DO  representation 

X  (x,  t;  uj)  =  x  (x,  t)  +  Yi  (f ;  uj)  Xi  (x,  t) 
into  Navier-Stokes  equations  we  obtain 

dp 

Cu  [x  (x,  t-  uj)  ;  u]  =  -  —  +  F0  +  YiFi  -  YYjFij  +  Zr  (f ;  uj)  pxr  (x,  t) 
dp 

Cx  [x  (x,  t,  uj)  ,  uj]  —  ~dy  jG ij  -t-  Zr  (t,  uj)  Pyr  (x,  t) 

dp 

Cu,  [x  (x,  t;  uj)  ;  uj]  =  -  —  +  H0  +  YiHi  -  YjYjHij  +  Zr  (f ;  uj)  pzr  (x,  t) 

Cp  [x  (x,  f;  w) ;  uj]  =  R0  +  YiR{  -  YjYjRij 
where  the  fields  on  the  right  hand  side  are  given  by 

1  d  (u2)  d  (uv)  d(uw)  ,  . 

F0  =  —A u  -  +  fv  +  rx  x,  t) 

Re  ax  ay  dz 
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fu  +  Ty  (x,  t) 


1  d(vu)  d  (y2)  d(vw) 

0  =  Re  ~  ~di  dy  dz 

1  .  d(wu )  d(wv)  d  (w2) 

H°=  -5T  -  “ST  “  &  -»?+T*(x'*) 


— 


9  (up)  d  (vp)  d  ( wp ) 


<9x 


dy 


dz 


and, 


p  _  1  An,  0<9( mu )  5  +  Uju)  3  +  ititu)  |  ^ 

■Pi  —  -p»  ^  o  o  0  i  jVi 

Ke  ox  ay  oz 


Gi  =  - 

Re 


d  (viU  +  Uiv)  nd(viv)  d(wiV  +  Viw) 


dx 


-  2 


dy 


dz 


~  fUi 


1  d(wiU  +  Uiw)  d(wiv  +  viiv)  d(wiw) 

Hi  =  R^Am - ^ - ^7 - 2  ^77 - 9  Pi 


Ri  =  uApi  - 

for  i=l, ...,  s,  and 


dx  dy  dz 

d  ( pin  +  Uip)  d  ( pp}  +  Vip)  d  (piiv  +  Wip) 


dx 


dy 


dz 


Fij  — 


Gij  = 


Hij  = 


Rij  — 


dx 

+ 

dy 

+ 

dz 

d(viUj) 

-  + 

d  (viVj) 

■  + 

d(viWj) 

dx 

dy 

dz 

d(wiUj) 

■  + 

d  (wiVj) 

i 

-  + 

d(wiWj ) 

dx 

dy 

dz 

d(piUj) 

-  + 

d(piVj) 

■  + 

d(piWj) 

with  i,  j  =  1, s. 

Moreover,  by  inserting  the  DO  representation  in  the  continuity  equation  we  obtain 


du  dv  dw 


+ -RT.  + +Yi(t;u;)  ^-  +  w-  +  ^-  -  0. 


dx  dy  dz 


duj  dv-,  dw; 


dx  dy  dz 


But  since  Yt  (t;  lo)  is  random  the  above  equation  has  the  equivalent  form 

du  dv  dw 

dx  dy  dz 

dui  dvi  dwi  . 

V)  H -x  V  — 0,  %—  l,...,s. 

dx  dy  dz 


A.l  Stochastic  pressure  field 

To  derive  an  equation  for  the  pressure  we  need  to  understand  its  role  in  the  stochastic 
context  of  the  operator  C  given  above.  Pressure  is  the  stochastic  quantity  which  guarantees 
that  for  every  possible  realization  ui  the  evolved  field  (u  (x,  t;  u) ,  v  (x,  t;  uj)  ,  w  (x,  t\  cj))  is 
incompressible.  Therefore,  the  stochastic  pressure  should  be  able  to  balance  all  the  non- 
divergent  contributions  from  the  terms  involved  in  the  operator  C.  To  this  end  we  choose 
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to  represent  the  stochastic  pressure  field  as 


p  =  Po  +  Yj,  (t;  to)  pi  -  Yj  (t;  u>)  Yj  (t;  u)  pij  +  Zr  (t;  u)  br 

Based  on  the  above  discussion,  the  mean  pressure  field  components  should  satisfy  the  fol¬ 
lowing  equation 

dF0  8G0  dH0 
APo  =  ^  +  ^  +  ^  =  0 


dx 


dy 


dz 


with  boundary  conditions  given  by 

(£,t;u)  =  0,  £  E dD\  and  po  =  const.  £  £dD2 

an 

The  stochastic  terms  in  C  multiplied  with  Yt  (t;  lo)  will  be  balanced  through  the  following 
equation 

.  dFi  dGi  dHi 

Api  =  -w—  +  =  0,  *  =  l,-,s 


dx  dy 


dz 


and  boundary  conditions 
dp. 


dn 


=  0,  £,  £dD i  and  pt  =  0,  £  €dD2,  *  =  1,...,s. 


Similarly,  for  the  stochastic  terms  multiplied  by  Yt  (f ;  lo)  Yj  ( t ;  lo)  we  will  have 


A  p^  = 


13  dx 
and  boundary  conditions 
dpij 


dFa  dGi 


dy 


+ 


OH, 


dz 


=  0,  i,j  =  l,...,s 


dn 


=  0,  £,  €dD i  and  pl3  =0,  £,  €dD2,  i,j  =  1 


Finally,  the  forcing  terms  will  be  balanced  through  the  family  of  equations 
Abr  =  ^  ^  ^  =  0,  r  =  1, ...,  R, 


dx 


dy 


dz 


with  boundary  conditions 

Qn 

— -  =  0,  £  €dDi  and  qr  =  0,  £,  &dD-2,  r  =  l,...,R. 

dn 

The  above  set  of  equations  guarantees  that  for  every  realization  lo  the  evolved  field  (u  (x,  t;u)  ,v  (x,  t;co),w  (x,  t 
will  always  be  incompressible.  Moreover,  the  evolution  operator  C  will  take  the  form 


Cu  [x  (x,  t;  u);u}  = 


~  YiYj 


Fft  +  Fv 

dx 


dpij 


dx 


+  Yi 
+  Fij 


+  f 

O  I  1  l 

OX 

+  Zr  ( t ;  lo) 


-Z^j  +  Vccr  (x,t) 


162 


Cv  [x  (x,  t;  u);w}  = 


dpo 

dy 


+  G0 


+  Yt 


-  YiYj 


dp, 


dy 


+  G 


dpi  ,  n 

r\  I  GJj, 

dy 

+  Zr  (t\  to) 


dqr  /  ,, 

-~d^  +  iPyr 


Cw  [x  (x,  t;  u);oj}  = 

-YiYj 


oz 


dpij 

dz 


+  Y 
+  Hij 


_^Pi  ,  rr 

OZ 

+  Zr  ( t ;  cj) 


dqr 

dz 


+  Ifiyr  (X,  t) 


Cp  [x  (x,  t;  u) ;  cu]  =  i?0  +  bi?j  -  bYji?, 


A. 2  Evolution  of  the  mean  field  u  (x,  t\uS) 

Using  the  corresponding  DO  equation  for  the  mean  we  obtain  the  set  of  deterministic  PDEs 

+  Fij\  +  tx  (x,  t) 


du  dpo 

—  ~  —— - r  r0  -  '^Yi(t)Yj(t) 


dt 


dx 


dpij 


dx 


dv_dp0  (  dp^  . 

7u  “  “ ~^y  +  G°  ~  CU(t)UW  +  Gij  1  +  Ty  ^  ^ 


dt 

dw 

~dt 


dpo 

dz 


+  H0-C 


dpij 

dz 


+  Hij  +  Tz  (x,  t) 


%  =  R{)-  CYi(t)Yj(t')Ri. 


du  dv  dw 
dx  dy  dz 


with  the  following  boundary  conditions 


u  (£,t;  u)  =  udDl  (£,i) ,  £  £dDi 

<9u 

—  (£,i;  w)  =  hdD2  (£,t) ,  £  edD-2 


dp 

dn 


(€,t-,w)  =  hPidD(t,t),  £edD 


A. 3  Evolution  of  the  stochastic  subspace  basis  Xi  (x,t;o;) 

We  will  first  calculate  the  quantity  E u  [C  [u  (x,  f; lo)  ;  w\  Yj  (i;  w)].  We  will  have 

dpn 


Eu  [Gu  [x  (x,  t;  u>) ;  w\  Yj  (t;  w)]  =  C Ym{t)Yj{t) 

-  M 
+  GYj(t)zr(t) 


dx 


Yj(t)Yrn(t)Yn(t) 

dqr 
dx 


-  +  En 

dpmn 
dx 

+  ipxr  (x,  t) 


+  Fn 
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[£v  [x  (x,  t;  u) ;  w]  Yj  (t;  w)]  =  Cym(t)y.(t) 


<9pn 


-  M 


n(t)yra(t)y„(i) 


-  +  Gj; 

dpmn 

dy 


+  G„ 


+  ^Yj(t)Zr(t) 


I  /  ^ 

-^7-  +  ^  (X,t) 


^  [£«  [x  (x,  i;  w) ;  w]  Y*  (i;  w)]  =  Cym(t)y.(f) 


dpr, 


9s: 


-  M, 


LYj(t)Ym(t)Yn(t) 
+  C  Yj(t)Zr(t) 


-  +  Hn 

dpmt 

dz 


+  Hn 


~~&z  +  ^ zr 


[£p  [x  (x,  t;  u>) ;  w]  1}  (t;  w)]  -  C Yrn(t)Yj(t)Rm  -  M^-(t)ym(t)yn(f)^ 


Multiplying  with  the  inverse  matrix  Cy^y  we  will  have 


Qui  =  Eu  [Cu  [x  (x,  t;  w) ;  w]  Y*  (t;  w)]  Cy^t)Yj{t) 


-dPi  +  F- 

o  I  rl 

OX 


~  CYi(t)Yj (■ t ) my  (t)yn(t) 


dffm? 

<9x 


+  Frr 


+  c 


-1 


Yi(t)Yj(tyJYj(t)zr{t ) 


-^7  +  ^r  (X,t) 


Qw  =  ^  [A;  [X  (x,  i;  w) ;  w]  Yj  (t;  w)] 


-§"+G, 

dy 


^Yi  (t)  Yj  (t)  Y  (t)  Ym(i)yn  (t) 


dpn 


dy 


+  Gv 


+  C 


-i 


Yi(t)Yj(t)^Yj(t)Zr(.t) 


9qr  .  i  ,  , 
~!hj  +  ipyr  (x’  ^ 


Qtw  =  ^  [£«  [x  (x,  t;  w) ;  w]  Y}  (t;  w)]  C^y  (t) 


r ^  I  -tli 

oz 


-  c 


-i 


Yj  (t)  Yj  (t)  ^Y  C t)Ym(t)Yn  (t) 


dpr 


dz 


+  H„ 


+  ^'Yi(t)YJ-(t)^'Y(*)Zr(t) 


9(lr  .  . 

--r^  +  Vzr  (X,t) 


Qpi  =  £"  [£p  [x  (x,  t-  u) ;  w]  Yj  (f;  w)]  Cy.^y. (<) 
=  Ri  ~  ^y,(i)rj(()^h(f)y™(f)y»(*)^“ 


Using  the  above  expressions  we  obtain  the  evolution  equations  for  the  basis  u*  (x,  t;  c 
dui 

=  Qui  ({Qwi?  ^m}  H“  ^m}  H“  Pm} ) 

dvi 

—  Qvi  ({ Qui:  Um}  Y  {Qvi:  Um}  +  {Qidi;  U>m}  +  {Qpii  Pm} )  Um 


),Pi  (x,t;w) 
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for  i 


dwi 

dt 


Qwi  {{Quii  Um{{  4"  {Qvii  ^m}  4"  {Qwii  Wm  }  4-  {  Q  pi  ,  Pm  }  )  wm 

dm  dVi  d'Wi 

0  =  — -  4 - -  4 - - 

dx  dy  dz 


&  Pi 

—  Q pi  ({ Quii  Um }  4~  {Qvii  Un }  4~  {Qwi  j  ^'rn  }  4“  {Qpi;  Pm})  Pm. 

1, s.  Moreover,  we  will  have  the  following  boundary  conditions 


u i  (^,t)  =  0, 


£eUD, 
£  G5Z42. 
£  €dD 


A. 4  Evolution  of  the  stochastic  coefficients  Y)  (x,  t;  cu) 


The  set  of  evolution  equations  for  the  stochastic  coefficients  will  take  the  form  of  a  SDE 


dt 


({Yj>  Tm }  4“  { 4~  { v'i ,  Hm }  -f-  { Pi ,  Rm } )  Un 

({Y?>  Rrnn  }  4"  { Vi ,  Ginn  }  4“  { iVj: ,  Hmn}  T  {  Pi  •  Rmn  } )  (  Yn  V£  ^Ym(t)Yn(t)) 


+  ({<£,„.  (X,  f)  ,  «;}  4-  (w  (x,  t)  ,  Uj}  4-  {<£V  (x,  t)  ,  Wj})  Zr  (f;  oj) 


{t)Yn(t)) 


Using  Gauss  theorem  we  have  for  every  scalar  field  a  and  every  divergent- free  vector  field 

/ 3 

J Va  (x)  ./3(x)  dx  =  j a  (0/3(0  -n  (0  d£ 

D  dD 

Applying  the  above  property  for  the  pressure  pm  and  the  vector  field  Uj  and  by  using 
(without  loss  of  generality)  the  boundary  conditions  of  section  5.2  we  obtain 


Ui 


dpm 

dx 


Vi , 


dptl 


dy 


+  (  w. 


dpm 

dz 


=  0. 


Furthermore,  by  applying  the  same  argument  for  the  pressure  fields  prnn  and  qk  and  the 
vector  fields  Uj  we  also  obtain 


J  dpmn 

\  dx 


+ 


Ui, 


dgr 

dx 


dpmn  1 

Vh~dy~  J 


+ 


dqr 

dy 


m, 


dpr 


Wi, 


dz 

dqr 


dz 


=  0 

=  0. 
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Hence,  the  stochastic  equation  describing  the  evolution  of  the  stochastic  coefficients  will  be 
take  the  form 

dYi 

jl  —  ( {  ^ ■  Fm}  “I-  F  {^ij  Hm}  H-  {Pii  Fm})  Ym 

{{Fi  Fnm  }  ,  Gmn  }  4~  {iCj,  Hmn  |  -f"  {  Pi  ■  Finn  } )  (YmYn  ^Ym(t)Yn(t )) 

+  ({^r  (x,  f)  ,  +  {(/?yr.  (x,  t) ,  Uj}  +  {<^r  (x,  t)  ,  Wj})  Zr  (f;  w)  . 
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Appendix  B 


Numerical  implementation  of  2D 
DO  for  Navier-Stokes 


In  the  following  paragraphs  we  will  give  a  brief  description  of  the  numerical  components  used 
in  the  computational  algorithm.  The  numerical  implementation  of  the  DO  field  equations 
for  the  description  of  stochastic  fluid  flows  (Chapter  5)  was  performed  in  Matlab.  In  Figure 
B-l  we  present  the  diagram  describing  the  computational  algorithm. 


B.l  Initial  conditions  formulation 


The  first  component  of  the  algorithm  is  the  formulation  of  the  initial  conditions.  This  is 
done  by  numerically  solving  the  eigenvalue  problem  of  Section  2.5  (Theorem  7).  We  use 
the  covariance  matrix  describing  the  spatial  stochastic  information  of  the  initial  conditions. 
After  spatial  discretization  of  the  domain  the  continuous  eigenvalue  problem  takes  the  form 
of  a  finite-dimensional  eigenvalue  problem,  with  symmetric  matrix  (symmetry  follows  from 
the  properties  of  the  covariance  matrix).  Note  that  for  a  domain  discretized  using  M 2  the 
size  of  the  matrix  to  be  diagonalized  is  M4.  Therefore  to  avoid  excessive  computational  cost 
it  is  essential  to  use  a  coarser  grid  for  the  diagonalization  of  the  covariance  matrix  and  then 
interpolate  on  the  finer  computational  grid. 

Through  the  diagonalization  procedure  we  obtain  a  set  of  eigenvectors  (u*o  (x) ,  i  = 
1, 2, ...)  and  the  corresponding  eigenvalues.  As  it  was  discussed  in  Section  2.5  the  eigenvalues 
represent  the  variance  along  the  direction  defined  by  the  corresponding  eigenvector.  By  a 
choosing  a  critical  amount  of  variance  below  which  uncertainty  is  considered  negligible  we 
obtain  the  initial  dimension  of  the  stochastic  subspace,  s. 

The  next  step  is  the  representation  of  the  stochastic  coefficients  Yoi  (w)  ,  i  =  1 . 2,  ...,s 
which  can  be  in  general  non-Gaussian.  To  represent  their  arbitrary  stochastic  structure  we 
use  a  random  number  generator  for  the  following  random  vector 

Yoi  M  =  (u0  (x;w) ,  ui0  (x)}  ,  i  =  1,2,  ...s. 

Note,  that  the  relatively  small  size  of  s  (i.e.  s  ~  10)  allows  for  the  use  of  a  large  number  of 
random  samples  to  represent  the  random  vector. 
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B.1.1  Storage  of  orthonormal  basis 


For  the  application  of  the  adaptive  criteria  it  is  essential  to  have  a  basis  that  will  be  able  to 
approximate  satisfactory  any  flow  realization.  The  eigenvectors  produced  by  the  diagonal- 
ization  of  the  initial  conditions  covariance  matrix  are  spatially  smooth  but  also  consist  an 
orthonormal  basis  that  can  approximate  (up  to  the  ability  given  by  the  computational  grid) 
any  flow  realization.  To  this  end  we  use  the  set  of  basis  elements  obtained  from  the  previous 
diagonalization  process.  Note  that,  assuming  the  initial  conditions  respect  the  boundary 
conditions  of  the  problem,  the  obtained  base  from  the  diagonalization  process  respects  the 
geometry  and  boundary  conditions  of  the  problem  as  well. 


B.2  Evolution  of  stochastic  state 

Having  the  initial  stochastic  conditions  for  the  DO  equations  we  may  now  proceed  for  their 
numerical  solution.  We  have  to  solve  a  stochastic  ODE  having  dimensionality  s  coupled  with 
sT  1  deterministic  PDEs  (see  Chapter  5).  For  the  spatial  discretization  of  the  deterministic 
PDEs  we  use  a  C-grid  in  combination  with  a  central  difference  scheme  (see  [54]).  For  the 
solution  of  the  involved  pressure  equations  we  use  the  Gauss-Seidel  method  also  described 
in  [54],  Time  discretization  is  performed  using  Euler’s  method  which  employs  first-order 
difference  quotients. 

For  the  solution  of  the  stochastic  differential  equation  we  evolve  each  one  of  the  random 
samples  (in  a  deterministic  sense)  created  previously  for  the  representation  of  the  stochastic 
coefficients  Yoi  (<*>) ,  i  =  1,2,  ...s.  Note  that  because  of  the  low  dimensionality,  s,  the  com¬ 
putational  cost  involved  in  the  solution  of  the  stochastic  differential  equation  is  negligible 
relative  to  the  evolution  of  the  s  +  1  deterministic  fields.  Using  the  statistics  of  the  evolved 
random  samples  we  are  able  to  compute  all  the  moments  required  for  the  evolution  of  the 
deterministic  fields. 

At  the  end  of  each  time  step  we  orthonormalize  the  stochastic  subspace  basis  elements 
Uj  (x,t) ,  i  =  1,2,  ...s.  Even  though  the  orthonormality  is  theoretically  preserved  by  the 
DO  equations  we  reenforce  it  numerically  at  the  end  of  each  time  step  to  avoid  round-off 
errors.  The  numerical  algorithm  that  we  use  is  the  modified  Gram-Schmidt  method  which 
guarantees  smaller  errors  in  finite-precision  arithmetic  (see  e.g.  [11]). 


B.3  Diagonalization  of  covariance  matrix  -  Adaptive  criteria 

We  evolve  the  stochastic  state  as  described  above  for  N  time  steps,  where  IV  is  a  small 
integer  number  (usually  chosen  between  10  an  100).  After  N  time  steps  the  solution  is 
expressed  in  the  special  reference  frame  where  the  the  stochastic  coefficients  Yt  (t;  u) ,  i  = 
1, 2,  ...s  are  uncorrelated.  This  is  done  by  diagonalizing  the  low- dimensional  matrix  C yy  and 
subsequently  rotating  the  stochastic  coefficients  and  the  stochastic  subspace  basis  elements 
as  described  in  Section  4.3.1. 

At  this  point  the  stochastic  state  of  the  system  is  expressed  in  suitable  form  so  that  the 
adaptive  criteria  can  be  applied  (Chapter  4).  Specifically,  the  diagonal  form  of  Cyy  allows 
for  the  removal  of  the  mode  with  the  smallest  variance  (if  the  corresponding  criteria  are 
satisfied)  or  the  addition  of  a  the  most  unstable  mode  (if  the  variance  of  the  modes  satisfies 
the  corresponding  criteria).  In  the  latter  case  the  computation  of  the  most  unstable  mode 
is  performed  using  the  orthonormal  base  computed  at  the  beginning  but  also  the  current 
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Diagonalization  of  ( 


Storage 
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Figure  B-l:  Computational  algorithm  for  the  adaptive  DO  equations. 
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state  of  the  system. 

After  the  application  of  the  adaptive  criteria  the  updated  form  of  the  system  state 
returns  to  the  ‘evolution’  component  of  the  algorithm. 

B.4  Storage  -  Plotting 

The  storage  and  plotting  is  performed  after  the  diagonalization  of  the  covariance  matrix 
and  the  rotation  of  the  stochastic  state  in  the  corresponding  reference  frame.  We  store  i) 
the  random  samples  for  the  stochastic  coefficients  Yt  ( t\u )  ,*  =  1,2,  ...s,  ii)  the  mean  field, 
and  iii)  the  stochastic  subspace  basis  elements  u*  (x,f)  ,  i  =  1,2,....  The  flow  fields  are 
visualized  using  either  the  streamfunction  or  the  vorticity  function.  The  samples  for  the 
stochastic  coefficients  are  used  to  obtain  the  corresponding  probability  density  function. 
This  is  done  using  a  kernel  smoothing  density  estimate  method  (see  [23])  that  allows  us  to 
obtain  smooth  probability  density  functions  of  general  form. 
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Appendix  C 

Normal  local  stability  of  general 
invariant  manifolds 


In  this  appendix  we  will  first  state  a  general  result  describing  the  normal  stability  properties 
of  invariant  manifolds  and  then  we  will  apply  it  to  generalize  the  results  presented  in  section 
6.2.2  for  the  stability  of  neutrally  buoyant  particles.  The  results  presented  in  the  current 
section  are  part  of  the  publication  [57] . 

Invariant  manifolds  are  distinguished  sets  of  trajectories  that  organize  the  global  ge¬ 
ometry  of  trajectories  in  a  dynamical  system.  Even  if  an  invariant  manifold  is  attracting 
(i.e.,  all  close  enough  trajectories  ultimately  converge  to  the  manifold),  it  may  admit  local¬ 
ized  regions  of  instability  where  nearby  trajectories  temporarily  depart  from  the  manifold. 
These  departures  and  later  returns  may  manifest  themselves  in  spectacular  jumps  along  the 
manifold.  Such  transient  jumps  are  often  important  to  locate  in  applications. 

In  this  section,  we  extend  the  ideas  from  the  specialized  setting  considered  in  section  6.2.2 
to  identify  localized  transverse  instability  and  attraction  on  general  invariant  manifolds.  Our 
main  tool  in  achieving  this  is  the  normal  infinitesimal  Lyapunov  exponent  (NILE),  defined 
as  the  leading  order  short-term  stretching  or  contraction  rate  at  points  of  an  invariant 
manifold.  We  derive  a  general  explicit  formula  for  the  NILE,  and  hence  for  the  normally 
stable  and  unstable  domains  of  an  invariant  manifold.  Subsequently,  we  apply  our  results 
to  characterize  the  stability  properties  of  the  dynamics  of  non-neutrally  buoyant  particles. 


C.l  Set-up  and  definitions 

Consider  a  dynamical  system  of  the  form 

X  =  F{X,t),  AeMn,  (C.l) 

with  n  >  2,  and  with  a  smooth  function  T :  Rn  xR->  Mn.  A  trajectory  X(t\  to,  Xq)  of  this 
system  is  a  solution  of  the  initial  value  problem  X(to)  =  Xq.  The  flow  map  associated  with 
(C.l)  is  a  two-parameter  family  of  transformations  defined  as 

F*0  :  Mn  -»•  Mn, 

Xq  e- >  X(t'to,Xo), 
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The  linearized  flow  map  DFf0  is  defined  as 

DFftQ  :  TRn  -»•  TRn, 

(P,v)  ^  (Fj;0(p),  DXoX(t-,t0,p)v)  , 

mapping  vectors  v  based  at  a  point  p  to  vectors  Dx0X(t;to,p)v  based  at  the  point  Ffo  (p) . 
The  notation  TW1  refers  to  the  tangent  bundle  of  Rn;  a  local  fiber  of  this  bundle  at  a  point 
p  G  Rn  will  be  denoted  by  TpMn. 

We  assume  that  M{t)  C  Mn  be  a  fc-dinrensional  differentiable  manifold  that  is  locally 
invariant  over  under  the  map  F(q  over  the  time  interval  This  means  that 

F/0(M(t0))cM(t),  (C.2) 

for  all  times  to,t  G  [ti, <2]  with  t  >  to- 

Beyond  invariance  and  smoothness,  we  make  no  further  assumption  about  the  manifold 
M{t).  We  do  note  that  M(t)  will  typically  only  be  of  interest  in  physical  applications  if  it 
is  robust,  i.e.,  persists  under  small  perturbations.  As  Fenichel  [42]  showed,  such  persistence 
is  ensured  if  M(t)  is  time-independent,  compact,  and  normally  hyperbolic. 


C.2  Normally  stable  and  normally  unstable  subsets 

Instead  of  the  asymptotic  behavior  along  M(t),  our  focus  here  is  the  instantaneous  behavior 
of  trajectories  close  to  M{t).  Specifically,  we  would  like  to  locate  points  on  M(t)  at  which 
most  transverse  perturbations  to  M(t)  start  growing  in  norm  over  infinitesimally  short 
times.  (A  measure  zero  set  of  transverse  perturbations  are  allowed  to  decrease  at  such 
points  due  to  the  presence  of  stable  directions.)  The  union  of  such  points  will  form  the 
normally  unstable  subset  Mu(t )  of  M(t).  Similarly,  the  normally  stable  subset  Ms(t )  of 
M(t)  is  the  union  of  points  in  M(t)  at  which  all  transverse  perturbations  to  M(t )  decay 
over  infinitesimally  short  times. 

Note  that  Ms(t)  and  Mu(t )  are  typically  not  invariant  subsets  of  M(t ):  trajectories 
in  M(t )  will  generally  either  avoid  both  of  these  subsets,  or  simply  cross  them.  Likewise, 
trajectories  near  M(t)  may  either  stay  away  from  the  subsets  Ms(t )  and  Mu(t )  of  M(t),  or 
pass  over  these  regions.  In  the  latter  case,  however,  the  passing  trajectories  will  break  away 
from  M(t)  while  near  Mu(t),  and  approach  M(t )  while  near  Ms{t )  (see  Fig.  C-l). 

If  Ms(t )  =  M(t),  then  M(t)  is  a  normally  hyperbolic  locally  invariant  manifold  that  only 
admits  stable  normal  directions;  in  this  case,  M(t)  has  an  n-dimensional  stable  manifold 
(or  domain  of  attraction).  If  Mu(t )  =  M(t),  then  M(t)  is  a  normally  hyperbolic  locally 
invariant  manifold  that  only  admits  unstable  normal  directions;  in  this  case,  M(t )  has  an 
n-dimensional  unstable  manifold. 

The  stable  and  unstable  subsets  described  above  have  a  major  impact  on  the  dynamics 
near  M(t).  As  it  turns  out  below,  locating  them  is  possible  from  explicit  calculations,  and 
hence  they  offer  a  feasible  alternative  to  Lyapunov-type  numbers  in  the  analysis  of  impact 
of  M(t)  on  phase  space  geometry.  We  make  these  ideas  more  precise  below. 


Definition  17  Let  TpM(t )  and  NpM(t)  denote  the  tangent  and  normal  space  of  M (t)  at 
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Figure  C-l:  Trajectories  jump  away  from  M (t)  over  the  unstable  subset  Mu(t ) ,  but  return  to 
M(t)  over  the  stable  subset  Ms(t) .  The  figure  assumes  that  M(t)  is  a  normally  hyperbolic 
attracting  manifold,  in  which  case  the  jumping  trajectory  keeps  approaching  the  same 
underlying  trajectory  on  M(t)  by  the  invariant  foliation  of  the  stable  manifold  Ws(M(t)). 


p  G  M(t),  respectively.  Also,  let 

n*  :  TpRn  =  TpM (t)  ©  NpM(t )  -»•  NpM{t ), 

(u,  V )  I— >  V, 


denote  the  natural  projection  from  TpRn,  the  tangent  space  o/R 71  at  p,  to  NpM(t).  We  then 
define  the  normal  infinitesimal  Lyapunov  exponent  (NILE)  at  a  point  p  6  M{t )  as 


&(p;  t) 


lim  -  log 

s^+0  S 


n 


t+s 

Ft+s 


(p) 


DF)+S 


NpM(t) 


(C.3) 


The  limit  in  the  definition  of  cr(p;  t)  exists  for  any  v  by  the  differentiability  of  the 
linearized  flow  map  DF fo  and  of  the  normal  projection  n^,t  ^  in  t.  Also  note  that  the  limit 

is  finite  because  DFf  =  I  by  definition,  and  hence  DF(+S  —  I  =  0(s).  The  action  of  the 
operator  DF(+S  on  a  vector  vq  €  NpM(t )  is  shown  in  Fig.  C-2. 

Next,  we  want  to  argue  that  exponent  a(p]  t )  is  independent  of  the  choice  of  the  trans¬ 
verse  bundle  in  which  we  seek  to  measure  growth  rates  from  M(t).  To  describe  this  inde¬ 
pendence  in  more  precise  terms,  we  let  NM(t )  be  another  smoothly  varying  ?r-dimensional 
vector  bundle  that  is  a  fibration  over  M(t);  we  require  this  bundle  to  be  transverse  to  M(t), 
by  which  we  mean  NpM(t )  ftl  TpM(t )  within  TpMn,  for  all  p  €  M{t). 

We  can  then  view  TpRn  as  the  direct  sum  of  NpM(t )  and  TpM(t), 

TpRn  =  NpM (t)  ©  TpM{t ), 

and  define  the  natural  projection  from  TpR"-  onto  NpM(t )  as 

%:  TpRn  =  TpM (t)  ©  NpM{t)  ->  NpM(t ), 

(u,  V)  e- >  v. 

We  require  each  fiber  NpM(t )  to  have  the  same  constant  position  relative  to  the  tangent 
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NpM(t) 


Nr,»wM(t+s) 


Figure  C-2:  The  operator  Ht^tfa^^DFj+s\ Np M(t)  maps  vectors  in  the  normal  space  NpM(t ) 
to  vectors  in  the  normal  space  ATFt+s ^ M(t  +  s).  The  NILE  a(p;  t )  is  the  exponential  rate  at 
which  the  norm  of  the  above  operator  grows  in  the  limit  of  infinitesimally  small  s.  Therefore, 
<r(p;  t)  measures  the  exponential  rate  at  which  the  normal  component  of  vectors  normal  to 
the  manifold  M(t)  grows  over  very  short  times.  (The  time  r  is  arbitrary  within  the  interval 
[t,t  +  s\.) 


space  TpM (t)  and  the  normal  space  and  normal  space  NpM(t).  Specifically,  we  require 
NM(t )  to  be  such  that  the  tangential  projection  from  NpM(t )  to  NpM (t), 

Q:  NPM (t)  ->  NpM{t), 

Q  =  n£|  NpM(t),  (C.4) 

is  independent  of  t  and  p.  This  latter  condition  is  important,  otherwise  the  value  of  the 
NILE  computed  on  NM(t)  would  be  affected  by  the  s-dependence  of  the  fiber  family 
NFi+S(p)AL(t  +  s )  along  the  trajectory  Ff+S(p)  starting  from  p  at  time  t. 

Proposition  18  If  we  define 

a(p\  t)  =  lim  -  log 
s^+0  S 

then  we  have  d(p ;  t )  =  <r(p;  t) 

Proof:  See  Haller  and  Sapsis,  2010  [57]. 

We  conclude  that  the  cr(p;  t)  is  independent  of  the  choice  of  transverse  directions  to  the 
invariant  manifold,  and  hence  gives  an  intrinsic  characterization  of  the  local  stability  of 
M (t)  at  t.  Motivated  by  this  observation,  we  introduce  the  following  definition  of  normally 
stable  and  unstable  subsets  of  M(t). 

Definition  19  We  define  the  normally  unstable  subset  of  M (t)  as 

Mu(t)  =  {p  e  M  |  cr(p;  f)  >  0}  , 


11*+*  BFj+s 
(p)  * 


NpM(t) 
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Figure  C-3:  The  manifold  M(t)  as  a  local  graph  over  the  x  variables. 


and  the  normally  stable  subset  of  M (t)  as 

Ms(t )  =  {p  G  M  I  cr(p;  t)  <  0}  . 

The  subsets  Mu(t )  and  Ms(t)"described  informally  above-are  now  defined  precisely 
through  the  Lyapunov-type  number  cr(p;t).  Unlike  the  Lyapunov-type  numbers  arising  in 
Fenichel’s  theory,  however,  cr(p]  t )  turns  out  to  be  computable  explicitly.  Consequently, 
Mu(t )  and  Ms(t )  can  be  identified  in  any  application,  provided  the  invariant  manifold  M(t) 
is  known  to  exist. 


C.3  Computing  the  NILE 

In  this  section,  we  give  two  alternative  expressions  that  can  be  used  for  computing  the 
NILE  in  specific  examples.  The  first  expression  will  assume  the  explicit  knowledge  of 
normal  vectors  along  the  manifold  Miff).  The  resulting  form  of  the  NILE  reveals  that— 
in  the  language  of  mechanics-<r(p,  t)  is  equal  to  the  maximal  normal  rate  of  strain  relative 
to  M(t )  at  the  point  p  G  M(t). 

The  second  expression  will  utilize  a  local  coordinate  representation  of  M(t ).  In  ap¬ 
propriate  coordinates  (x,y)  G  x  Mn~fc,  the  k- dimensional  invariant  manifold  M(t )  can 
always  be  written  locally  as  a  smooth  graph: 

M(t)  =  j(x,  y)  G  Rk  X  :  y  =  <p(x,t) ,  x  €  D(t)  j,  i  G  [ti,  *2] ;  (C.5) 

here  D(t )  is  an  open  domain  on  which  the  local  coordinate  x  is  defined  at  time  t  (see  Fig. 
C-3).  The  vector  variable  x  may  contain  some  parameters  (dummy  variables)  on  which 
the  dynamical  system  depends;  this  is  why  we  have  suppressed  any  explicit  parameter 
dependence  in  (C.l)  and  (C.6). 

In  the  local  variables  (x,y),  the  dynamical  system  (C.l)  takes  the  form 

x  =  f(x,y,t), 

y  =  g(x,y,t ),  (C.6) 

where  /  and  g  are  sufficiently  smooth  functions  of  their  arguments. 
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Theorem  20  (i)  Let  n(p,t)  be  a  unit  normal  vector  field  to  M{t),  that  is  smooth  both  in 
p  €  M(t)  and  in  t.  We  then  have 


cr(p,t.)  =  max  (n(p,t),  DT(p,t)n(p,t))  , 
n(p,t)eNpM(t) 


(ii) 


with  (  •  ,  •  )  denoting  the  standard  Euclidean  inner  product. 

Using  the  local  form  y  =  <p(x,t)  of  M{t)  and  the  vector  field  (C.6),  we  define  the  matrix 
field 

r(x,  t)  =  gy(x,  p  (. x ,  t) ,  t)  -  tpx(x,  t)fy(  x,  p  (x,  t),t),  ( C.7 ) 

where  the  subscript  refers  to  differentiation  with  respect  to  the  variable  in  the  subscript. 
We  then  have 


a(x,t)  =  Amax  [r(x,f)  +  rT(x,t)]  /2, 
where  Amax  [  •  ]  refers  to  the  largest  eigenvalue  of  a  symmetric  matrix. 
Proof:  See  Haller  and  Sapsis,  2010  [57]. 


C.4  Locating  stable  and  unstable  neighborhoods  of  M(t) 

The  normally  stable  and  unstable  subsets  partition  M(t)  into  subsets  in  which  infinitesi¬ 
mally  small  perturbations  transverse  to  M{t)  start  decaying  or  growing,  respectively.  The 
norm  of  such  small  perturbations  is  instantaneously  constant  on  the  boundary  between 
Ms(t )  and  Mu(t). 

As  trajectories  leave  the  vicinity  of  M(t),  the  growth  of  their  distance  is  no  longer 
captured  accurately  by  the  linearized  flow  along  M(t).  To  locate  finite-size  neighborhoods 
of  M(t)  where  trajectories  increase  their  distances  from  M(t ),  we  again  assume  that  M(t) 
is  given  locally  in  the  form  of  a  graph  (C.5).  As  in  the  proof  of  Theorem  20,  we  use  the 
change  of  coordinates 

z  =  y-p(x,t) 

to  transform  M(t)  to  the  z  =  0  plane.  In  the  transformed  coordinates,  the  flow  satisfies 
x  =  f  (x,z  +  p  (x,  t.)  ,t) , 

z  =  g  (x,  z  +  tp  (x,t)  ,t)  -  (px  (x,  t )  /  (x,z  +  ip  (x,  t)  ,t)  -  pt  (x,  t ) . 

Taking  the  inner  product  of  the  second  equation  with  z  gives 

\~fft\z\2  =  (s(x,z  +  ip(x,t)  ,t)  -ipx  (x,t)  f  (x,z  +  ip(x,t)  ,t)  -  pt  (x,  t) ,  z)  . 

The  boundary  between  domains  of  instantaneous  growth  from,  and  decay  to,  the  {z  =  0} 
plane  is  given  by  the  instantaneous  stability  boundary 


B(t)  =  {(x,  z)  :  (g(x,z  +  ip  (x,  t)  ,t)  -  px  (x,  t)  f  (x,z  +  ip  (x,  t )  ,t)  -  <pt  (x,  t ) ,  z) 
The  stable  and  unstable  neighborhoods  of  M(t)  are  given  by 


0}. 

(C.8) 


S(t)  =  {{x,z)  :  {g(x,z  +  (p(x,t)  ,t)  -  px  (x,  t)  f  (x,z  +  <p  (x,  t)  ,t)  -  (pt(x,t) ,  z)  <  0}  , 
U (■ t )  =  {(x,  z)  :  (g(x,z  +  <p  (x,  t)  ,t)  -  p>x  (x,  t)  f  (x,z  +  p>  (x,  t)  ,t)  -  <pt  (x,  t)  ,  z)  >  0}  . 
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Note  that 


(g  (x,z  +  p  (. x ,  f) ,  t)  -  (Ar  (®,  *)  /  (x,z  +  <p  (x,  t )  ,f)  -  ipt  (x,  t ) ,  2) 

=  (by  (a:,  *),*)  —  Az  (®,  i)  /j,  (z,A  (x,  t)  ,t)]  z,z)  +  0  (jz|3) 

=  bl2  (^[T{x,t) +  VT(x,t)\ez,ez^  +  0(\z\)  ,  (C.9) 

therefore 

M(t)  n  S(t)  =  j(x,z)  :  z  =  0,  Amax  *  [T(x,t)  +  TT(x,f)]  <o|, 

M(t)  fl  U(t)  =  j(x,z)  :  2;  =  0,  Amax  ^  [r(x,t)  +rT(x,f)]  >o|. 

As  a  result,  we  conclude  that 

Ms(t)  =  M(t)  n  S(t), 

Mu(t)  C  M(t)  n  U(t). 

Also  note  that  by  (C.8)  and  (C.9),  both  the  instantaneous  stability  boundary  B(t)  and  the 
cylindrical  surface 

B(t)  =  {(x,z):  a(x,t)  =  0}  (C.10) 

have  the  same  intersection  with  the  manifold  M(t).  Therefore,  near  the  manifold  M(t),  the 
condition  a(x,t)  =  0  gives  an  approximation  to  the  instantaneous  stability  boundary  B(t)\ 
the  error  of  the  approximation  grows  linearly  in  the  order  of  the  distance  from  the  manifold. 
This  error  can  be  decreased  to  quadratic  in  the  distance  if  we  also  include  the  O  (jz|3)  term 
in  (C.9)  in  our  calculations.  In  that  case,  the  calculated  dividing  surface  B(t)  will  have  a 
quadratic  tangency  with  B(t)  along  M{t). 

The  instantaneous  stability  boundary  B(t)  between  stable  and  unstable  neighborhoods 
of  M (t)  can  also  be  computed  directly  from  (C.8)  without  Taylor  expansion  in  the  z  variable. 
Specifically,  for  any  fixed  z  value,  we  can  locate  the  curve  of  x  values  satisfying  (C.8). 
Putting  all  these  curves  together,  we  obtain  the  surface  B(t). 


C.5  Stability  properties  of  non-neutrally  buoyant  particles 


Note  that  (6.3)  is  of  the  form  (C.6)  with  x  =  x  E  M2,  g  =  v  €  M2,  and  with 


/(x,v,f)  =  v, 

1  /  m  3RDu(x,  t.) 

<?(x,  v,  t)  =  -  -  [v  -  u(x,  t)}  +  - - ^ 


/  3R\ 

+  P-t)* 


(C.ll) 


In  chapter  6  we  proved  that  for  any  fixed  e  >  0  small  enough,  equation  (6.3)  admits  a 
globally  attracting  two-dimensional  invariant  manifold  of  the  form 

M(t)  =  |(x,v)  e  M4  :  v  =  u(x,t)  +  e  ^-1  ^  ~  g  +C’(e2)}-  (C-12) 
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By  comparison  with  the  general  form  (C.5),  formula  (C.12)  yields 


¥>(x,i)  =  u(x,f)  +  e  —  -  1 


Hu  (x,  t) 


-g  +  C(£2) 


(C.13) 


We  apply  Theorem  20  to  identify  the  stable  and  unstable  subsets  of  M(t).  Using  (C.ll) 
and  (C.13),  we  obtain  that  the  matrix  r(x,t)  satisfies 

r(x,  t)  =  [gv  —  y’x/v]v=^(x,t) 


= - /  -  <£x 

e 


=  --/-Vu(x,i)  +  0(e), 


and  hence  the  NILE  can  be  written  as 


cr(x,f)  =  Amax  [rT(x,  t)  +T(x,i)]  /2 

—  ^max  —I  2  ^Vx  (x>  t')  T  V^x  (X)  t)  ^ 


+  ^Amax  Vu(x,i)  +  Vu(x,  t)T  +0(e) 


Here  we  used  the  fact  that  by  incompressibility  of  the  flow,  the  two-dimensional  symmetric 
matrix  Vu  (x,  t)  +  Vu  (x,  t)T  has  zero  trace;  as  a  result,  its  eigenvalues  have  opposite  signs. 
Up  to  an  order  O  (e)  error,  therefore,  the  stable  and  unstable  subsets  of  M (t)  satisfy 


Ms(t)  «  <  (x,v)  G  M(t)  :  An 


Mu(t)  «  <  (x,v)  G  M(t)  :  An 


Vu  (x,t)  +  Vu(x,i)J  ^  1 
2  <  e 


Vu  (x,  t)  +  Vu  (x,  t)1  l[ 

2  >  e  I' 


(C.14) 
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